P proti NP: pregled Scotta Aaronsona

Vir, odlomek za odlomkom

1. u1 P ?= NP
2. u2 Scott Aaronson∗
3. u3 Abstract
4. u4 In 1955, John Nash sent a remarkable letter to the National Security Agency, in which— seeking to build theoretical foundations for cryptography—he all but formulated what today
5. u5 ?= NP problem, considered one of the great open problems of science. survey the status of this problem in 2017, for a broad audience of mathematicians, scientists,
6. u6 I offer a personal perspective on what it’s about, why it’s important, why it’s reasonable to conjecture that P̸ = NP is both true and provable, why proving it is so hard, the landscape of related problems, and crucially, what progress has been made in the last half-
7. u7 The discussion of progress includes diagonalization and circuit lower bounds; the relativization, algebrization, and natural proofs barriers; and the recent works of Ryan Williams and Ketan Mulmuley, which (in different ways) hint at a duality between impossibility proofs and algorithms.
8. u8 Contents
9. u9 1
10. u10 Introduction
11. u11 3
12. u12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13. u13 4
14. u14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15. u15 6
16. u16 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17. u17 6
18. u18 . . . . . . . . . . . . . . . . . . . . . . . . .
19. u19 7
20. u20 The Kitchen-Sink Objection . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21. u21 7
22. u22 The Mathematical Snobbery Objection
23. u23 8
24. u24 The Sour Grapes Objection . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25. u25 8
26. u26 The Obviousness Objection . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27. u27 9
28. u28 The Constructivity Objection . . . . . . . . . . . . . . . . . . . . . . . . . . .
29. u29 9
30. u30 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31. u31 10
32. u32 Formalizing P ?= NP and Central Related Concepts 2
33. u33 10
34. u34 NP-Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35. u35 12
36. u36 Other Core Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37. u37 16
38. u38 Search, Decision, and Optimization . . . . . . . . . . . . . . . . . . . . . . . .
39. u39 16
40. u40 The Twilight Zone: Between P and NP-complete . . . . . . . . . . . . . . . .
41. u41 17
42. u42 . . . . . . . . . . . . . . . . . . . . . . .
43. u43 17
44. u44 Supported by a Vannevar Bush / NSSEFF Fellowship from the US Department of Defense. Much of this work was done while the author was supported by an NSF Alan T. Waterman award.
45. u45 Factoring and Graph Isomorphism . . . . . . . . . . . . . . . . . . . . . . . .
46. u46 19
47. u47 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
48. u48 20
49. u49 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
50. u50 21
51. u51 Beyond Polynomial Resources . . . . . . . . . . . . . . . . . . . . . . . . . . .
52. u52 22
53. u53 Beliefs About P ?= NP 3
54. u54 24
55. u55 Independent of Set Theory? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
56. u56 26 4
57. u57 Why Is Proving P̸ = NP Difficult?
58. u58 29 5
59. u59 Strengthenings of the P̸ = NP Conjecture
60. u60 31
61. u61 Different Running Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
62. u62 31
63. u63 Nonuniform Algorithms and Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . .
64. u64 32
65. u65 Average-Case Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
66. u66 34
67. u67 Cryptography and One-Way Functions . . . . . . . . . . . . . . . . . . . . . .
68. u68 35
69. u69 Randomized Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
70. u70 37
71. u71 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
72. u72 38
73. u73 Quantum Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
74. u74 39 6
75. u75 Progress
76. u76 40
77. u77 Logical Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
78. u78 43
79. u79 Circuit Lower Bounds Based on Counting . . . . . . . . . . . . . . . . . . . .
80. u80 44
81. u81 The Relativization Barrier . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
82. u82 46
83. u83 Combinatorial Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
84. u84 47
85. u85 Proof Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
86. u86 48
87. u87 Monotone Circuit Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . .
88. u88 49
89. u89 Small-Depth Circuits and the Random Restriction Method
90. u90 50
91. u91 Small-Depth Circuits and the Polynomial Method
92. u92 53
93. u93 The Natural Proofs Barrier . . . . . . . . . . . . . . . . . . . . . . . . . . . .
94. u94 54
95. u95 Arithmetization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
96. u96 58
97. u97 IP = PSPACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
98. u98 58
99. u99 Hybrid Circuit Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . .
100. u100 61
101. u101 The Algebrization Barrier . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
102. u102 63
103. u103 Ironic Complexity Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
104. u104 64
105. u105 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
106. u106 65
107. u107 NEXP̸ ⊂ACC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
108. u108 68
109. u109 Arithmetic Complexity Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
110. u110 72
111. u111 Permanent Versus Determinant . . . . . . . . . . . . . . . . . . . . . . . . . .
112. u112 74
113. u113 Arithmetic Circuit Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . .
114. u114 77
115. u115 Arithmetic Natural Proofs? . . . . . . . . . . . . . . . . . . . . . . . . . . . .
116. u116 81
117. u117 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
118. u118 84
119. u119 From Complexity to Algebraic Geometry
120. u120 86
121. u121 . . . . . . . . . . . . . . . . . . . . . . . . .
122. u122 87
123. u123 The Quest for Obstructions . . . . . . . . . . . . . . . . . . . . . . . . . . . .
124. u124 88
125. u125 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
126. u126 91
127. u127 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
128. u128 92
129. u129 The Lessons of GCT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
130. u130 95
131. u131 The Only Way? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
132. u132 97 7
133. u133 Conclusions
134. u134 99 8
135. u135 Acknowledgments
136. u136 102 9
137. u137 Appendix: Glossary of Complexity Classes
138. u138 119
139. u139 1
140. u140 Introduction
141. u141 “Now my general conjecture is as follows: for almost all sufficiently complex types of
142. u142 enciphering, especially where the instructions given by different portions of the key
143. u143 interact complexly with each other in the determination of their ultimate effects on the
144. u144 enciphering, the mean key computation length increases exponentially with the length
145. u145 of the key, or in other words, the information content of the key ... The nature of this
146. u146 conjecture is such that I cannot prove it, even for a special type of ciphers.
147. u147 expect it to be proven.” —John Nash, 1955 [202]
148. u148 In 1900, David Hilbert challenged mathematicians to design a “purely mechanical procedure” to determine the truth or falsehood of any mathematical statement.
149. u149 That goal turned out to be impossible.
150. u150 But the question—does such a procedure exist, and why or why not?—helped launch two related revolutions that shaped the twentieth century: one in science and philosophy, as the results of G¨odel, Church, Turing, and Post made the limits of reasoning itself a subject of mathematical analysis; and the other in technology, as the electronic computer achieved, not all of Hilbert’s dream, but enough of it to change the daily experience of most people on earth.
151. u151 Although there’s no “purely mechanical procedure” to determine if a mathematical statement S is true or false, there is a mechanical procedure to determine if S has a proof of some bounded length n: simply enumerate over all proofs of length at most n, and check if any of them prove S.
152. u152 The P ?= NP problem asks whether there’s a fast This method, however, takes exponential time.
153. u153 algorithm to find such a proof (or to report that no proof of length at most n exists), for a suitable meaning of the word “fast.” One can think of P ?= NP as a modern refinement of Hilbert’s 1900 question.
154. u154 The problem was explicitly posed in the early 1970s in the works of Cook and Levin, though versions were stated earlier—including by G¨odel in 1956, and as we see above, by John Nash in 1955.
155. u155 Think of a large jigsaw puzzle with (say) 101000 possible ways of arranging the pieces, or an encrypted message with a similarly huge number of possible decrypts, or an airline with astronomically many ways of scheduling its flights, or a neural network with millions of weights that can be set independently. All of these examples share two key features:
156. u156 (1) a finite but exponentially-large space of possible solutions; and
157. u157 (2) a fast, mechanical way to check whether any claimed solution is “valid.” (For example, do
158. u158 the puzzle pieces now fit together in a rectangle? Does the proposed airline schedule achieve
159. u159 the desired profit? Does the neural network correctly classify the images in a test suite?)
160. u160 We’re asking whether, under the above conditions, there’s a general method to find a valid solution whenever one exists, and which is enormously faster than just trying all the possibilities one by one, from now till the end of the universe, like in Jorge Luis Borges’ Library of Babel.
161. u161 Notice that Hilbert’s goal has been amended in two ways.
162. u162 On the one hand, the new task is “easier” because we’ve restricted ourselves to questions with only finitely many possible answers, each of which is easy to verify or rule out. On the other hand, the task is “harder” because we now insist on a fast procedure: one that avoids the exponential explosion inherent in the brute-force approach.
163. u163 Of course, to discuss such things mathematically, we need to pin down the meanings of “fast”
164. u164 ?= NP question corresponds to one and “mechanical” and “easily checked.”
165. u165 natural choice for how to define these concepts, albeit not the only imaginable choice.
166. u166 For the impatient, P stands for “Polynomial Time,” and is the class of all decision problems (that is, infinite sets of yes-or-no questions) solvable by a standard digital computer—or for concreteness, a Turing machine—using a polynomial amount of time. By “polynomial time,” we mean that the machine uses a number of steps that’s upper-bounded by the length of the input question (i.e., the number of bits needed to write it down) raised to some fixed power, as opposed (say) to growing exponentially with the length. Meanwhile, NP stands for “Nondeterministic Polynomial Time,” and is the class of all decision problems for which, if the answer is “yes,” then there’s a polynomial-size proof that a Turing machine can verify in polynomial time.
167. u167 It’s immediate that P ⊆NP, so the question is whether this containment is proper (and hence P̸ = NP), or whether NP ⊆P (and hence P = NP).
168. u168 ?= NP 1.1
169. u169 The Importance of P Before getting formal, it seems appropriate to say something about the significance of the P ?= NP
170. u170 ?= NP, we might say, shares with Hilbert’s original question the character of a question.
171. u171 “math problem that’s more than a math problem”: a question that reaches inward to ask about mathematical reasoning itself, and also outward to everything from philosophy to natural science to practical computation.
172. u172 To start with the obvious, essentially all the cryptography that we currently use on the Internet— for example, for sending credit card numbers—would be broken if P = NP (and if, moreover, the algorithm were efficient in practice, a caveat we’ll return to later). Though he was writing 16 years before P ?= NP was explicitly posed, this is the point Nash was making in the passage with which we began.
173. u173 The reason is that, in most cryptography, the problem of finding the decryption key is an NP search problem: that is, we know mathematically how to check whether a valid key has been found. The only exceptions are cryptosystems like the one-time pad and quantum key distribution, which don’t rely on any computational assumptions (but have other disadvantages, such as the need for huge pre-shared keys or for special communication hardware).
174. u174 It was articulated, for example, in Kurt G¨odel’s now-famous 1956 letter to John von Neumann, which sets out what we now call the P ?= NP question. G¨odel wrote:
175. u175 If there actually were a machine with [running time] ∼Kn (or even only with ∼Kn2)
176. u176 [for some constant K independent of n], this would have consequences of the greatest
177. u177 That is to say, it would clearly indicate that, despite the unsolvability of
178. u178 the Entscheidungsproblem [Hilbert’s problem of giving a complete decision procedure
179. u179 for mathematical statements], the mental effort of the mathematician in the case of yes-
180. u180 or-no questions could be completely [added in a footnote: apart from the postulation
181. u181 of axioms] replaced by machines. One would indeed have to simply select an n so large
182. u182 that, if the machine yields no result, there would then also be no reason to think further
183. u183 Expanding on G¨odel’s observation, some modern commentators have explained the importance of P ?= NP as follows. It’s well-known that P ?= NP is one of the seven Clay Millennium Problems (alongside the Riemann Hypothesis, the Yang-Mills mass gap, etc.), for which a solution commands a million-dollar prize [74].
184. u184 For if someone discovered that P = NP, and if moreover the algorithm was efficient in practice, that person could solve not merely one Millennium Problem but all seven of them—for she’d simply need to program her computer to search for formal proofs of the other six conjectures.1 Of course, if (as most computer scientists believe) P̸ = NP, a proof of that would have no such world-changing implications, but even the fact that such a proof could rule out those implications underscores the enormity of what we’re asking.
185. u185 While theoretical computer scientists (including me!)
186. u186 ?= NP is not quite equivalent to the questions of have not always been above poetic flourish, P “whether human creativity can be automated,” or “whether anyone who can appreciate a symphony is Mozart, anyone who can recognize a great novel is Jane Austen.” Apart from the obvious point that no purely mathematical question could fully capture these imponderables, there are also more specific issues.
187. u187 For one thing, while P ?= NP has tremendous relevance to artificial intelligence, it says nothing about the differences, or lack thereof, between humans and machines.
188. u188 Indeed, P̸ = NP would represent a limitation on all classical digital computation, one that might plausibly apply to human brains just as well as to electronic computers. Nor does P̸ = NP rule out the possibility of robots taking over the world. To defeat humanity, presumably the robots wouldn’t need to solve arbitrary NP problems in polynomial time: they’d merely need to be smarter than us, and to have imperfect heuristics better than the imperfect heuristics that we picked up from a billion years of evolution! Conversely, while a proof of P = NP might hasten a robot uprising, it wouldn’t guarantee one. For again, what P ?= NP asks is not whether all creativity can be automated, but only creativity whose fruits can quickly be verified by computer programs.
189. u189 To illustrate, suppose we wanted to program a computer to create new Mozart-quality symphonies and Shakespeare-quality plays. If P = NP via a practical algorithm, then these feats would reduce to the seemingly easier problem of writing a computer program to recognize great works of art.
190. u190 And interestingly, P = NP might also help with the recognition problem: for example, by letting us train a neural network that reverse-engineered the expressed artistic preferences of hundreds of human experts.
191. u191 But how well that neural network would perform is an empirical
192. u192 1Here we’re using the observation that, once we fix a formal system (say, first-order logic plus the axioms of ZF set theory), deciding whether a given statement has a proof at most n symbols long in that system is an NP problem, which can therefore be solved in time polynomial in n assuming P = NP.
193. u193 We’re also assuming that the other six Clay conjectures have ZF proofs that are not too enormous: say, 1012 symbols or fewer, depending on the exact running time of the assumed algorithm.
194. u194 In the case of the Poincar´e Conjecture, this can almost be taken to be a fact, modulo the translation of Perelman’s proof [209] into the language of ZF.
195. u195 question outside the scope of mathematics.
196. u196 ?= NP 1.2
197. u197 Objections to P After modest exposure to the P ?= NP problem, some people come up with what they consider an irrefutable objection to its phrasing or importance.
198. u198 Since the same objections tend to recur, in this section I’ll collect the most frequent ones and make some comments about them.
199. u199 1.2.1
200. u200 The Asymptotic Objection Objection: P ?= NP talks only about asymptotics—i.e., whether the running time of an algorithm grows polynomially or exponentially with the size n of the question that was asked, as n goes to infinity. It says nothing about the number of steps needed for concrete values of n (say, a thousand or a million), which is all anyone would ever care about in practice.
201. u201 Response: It was realized early in the history of computer science that “number of steps” is not a robust measure of hardness, because it varies too wildly from one machine model to the next (from Macs to PCs and so forth), and also depends heavily on low-level details of how the problem is encoded.
202. u202 The asymptotic complexity of a problem could be seen as that contribution to its hardness that is clean and mathematical, and that survives the vicissitudes of technology. Of course, real-world software design requires thinking about many non-asymptotic contributions to a program’s efficiency, from compiler overhead to the layout of the cache (as well as many considerations that have nothing to do with efficiency at all).
203. u203 But any good programmer knows that asymptotics matter as well.
204. u204 More specifically, many people object to theoretical computer science’s identification of “polynomial” with “efficient” and “exponential” with “inefficient,” given that for any practical value of n, an algorithm that takes 1.0000001n steps is clearly preferable to an algorithm that takes n1000 steps.
205. u205 This would be a strong objection, if such algorithms were everyday phenomena.
206. u206 Empirically, however, computer scientists found that there is a strong correlation between “solvable in polynomial time” and “solvable efficiently in practice,” with most (but not all) problems in P that they care about solvable in linear or quadratic or cubic time, and most (but not all) problems outside P that they care about requiring cn time via any known algorithm, for some c significantly larger than 1.
207. u207 Furthermore, even when the first polynomial-time algorithm discovered for some problem takes (say) n6 or n10 time, it often happens that later advances lower the exponent, or that the algorithm runs much faster in practice than it can be guaranteed to run in theory.
208. u208 This is what happened, for example, with linear programming, primality testing, and Markov Chain Monte Carlo algorithms.
209. u209 Having said that, of course the goal is not just to answer some specific question like P ?= NP, but to learn the truth about efficient computation, whatever it might be. If practically-important NP problems turn out to be solvable in n1000 time but not in n999 time, or in 1.0000001n time,
210. u210 From this perspective, one could argue that P ?= NP simply serves as a marker of then so be it.
211. u211 ignorance: in effect we’re saying, “if we can’t even answer this, then surely we can’t answer the more refined questions either.”
212. u212 1.2.2
213. u213 The Polynomial-Time Objection Objection: But why should we draw the border of efficiency at the polynomial functions, as opposed to any other class of functions—for example, functions upper-bounded by n2, or functions of the form nlogc n (called quasipolynomial functions)?
214. u214 Response: There’s a good theoretical answer to this: it’s because polynomials are the smallest class of functions that contains the linear functions, and that’s closed under basic operations like addition, multiplication, and composition. For this reason, they’re the smallest class that ensures that we can compose “efficient algorithms” a constant number of times, and still get an algorithm that’s efficient overall.
215. u215 For the same reason, polynomials are also the smallest class that ensures that our “set of efficiently solvable problems” is independent of the low-level details of the machine model.
216. u216 Having said that, much of algorithms research is about lowering the order of the polynomial, for problems already known to be in P, and theoretical computer scientists do use looser notions like quasipolynomial time whenever they’re needed.
217. u217 1.2.3
218. u218 The Kitchen-Sink Objection
219. u219 ?= NP is limited, because it talks only about discrete, deterministic algorithms
220. u220 Objection: P
221. u221 that find exact solutions in the worst case—and also, because it ignores the possibility of natural processes that might exceed the limits of Turing machines, such as analog computers, biological computers, or quantum computers.
222. u222 Response: For every assumption mentioned above, there’s now a major branch of theoretical computer science that studies what happens when one relaxes the assumption: for example, randomized algorithms, approximation algorithms, average-case complexity, and quantum computing. I’ll discuss some of these branches in Section 5. Briefly, though, there are deep reasons why many of these ideas are thought to leave the original P ?= NP problem in place. For example, according to the P = BPP conjecture (see Section 5.4.1), randomized algorithms yield no more power than P, while careful analyses of noise, energy expenditure, and the like suggest that the same is true for analog computers (see [3]).
223. u223 Meanwhile, the famous PCP Theorem and its offshoots (see Section 3) have shown that, for many NP problems, there can’t even be a polynomial-time algorithm to approximate the answer to within a reasonable factor, unless P = NP.
224. u224 In other cases, new ideas have led to major, substantive strengthenings of the P̸ = NP conjecture (see Section 5): for example, that there exist NP problems that are hard even on random inputs, or hard even for a quantum computer.
225. u225 Obviously, proving P̸ = NP itself is a prerequisite to proving any of these strengthened versions.
226. u226 There’s one part of this objection that’s so common that it requires some separate comments. Namely, people say that even if P̸ = NP, in practice we can almost always find good enough solutions to the problems we care about, for example by using heuristics like simulated annealing or genetic algorithms, or by using special structure or symmetries in real-life problem instances.
227. u227 Certainly there are cases where this assumption is true. But there are also cases where it’s false: indeed, the entire field of cryptography is about making the assumption false! In addition, I believe our practical experience is biased by the fact that we don’t even try to solve search problems that we “know” are hopeless—yet that wouldn’t be hopeless in a world where P = NP (and where the algorithm was efficient in practice).
228. u228 For example, presumably no one would try using brute-force search to look for a formal proof of the Riemann Hypothesis one billion lines long or shorter, or a
229. u229 10-megabyte program that reproduced most of the content of Wikipedia within a reasonable time (possibly needing to encode many of the principles of human intelligence in order to do so).
230. u230 Yet both of these are “merely” NP search problems, and things one could seriously contemplate in a world where P = NP.
231. u231 1.2.4
232. u232 The Mathematical Snobbery Objection Objection: P ?= NP is not a “real” math problem, because it talks about Turing machines, which are arbitrary human creations, rather than about “natural” mathematical objects like integers or manifolds.
233. u233 Response: The simplest reply is that P ?= NP is not about Turing machines at all, but about algorithms, which seem every bit as central to mathematics as integers or manifolds.
234. u234 Turing machines are just one particular formalism for expressing algorithms, as the Arabic numerals are one formalism for integers. And just like the Riemann Hypothesis is still the Riemann Hypothesis in base-17 arithmetic, so essentially every formalism for deterministic digital computation ever proposed gives rise to the same complexity classes P and NP, and the same question about whether they’re equal. (This observation is known as the Extended Church-Turing Thesis.)
235. u235 This objection might also reflect lack of familiarity with recent progress in complexity theory, which has drawn on Fourier analysis, arithmetic combinatorics, representation theory, algebraic geometry, and dozens of other subjects about which yellow books2 are written. Furthermore, in Section 6.6, we’ll see Geometric Complexity Theory (GCT): a staggeringly ambitious program for proving P̸ = NP that throws almost the entire arsenal of modern mathematics at the problem, including geometric invariant theory, plethysms, quantum groups, and Langlands-type correspondences—and that relates P ?= NP, at least conjecturally, to other questions that mathematicians have been trying to answer for a century. Even if GCT’s specific conjectures don’t pan out, they illustrate how progress toward proving P̸ = NP could involve deep insights from many parts of mathematics.
236. u236 1.2.5
237. u237 The Sour Grapes Objection Objection: P ?= NP is so hard that it’s impossible to make anything resembling progress on it, at least at this stage in human history—and for that reason, it’s unworthy of serious effort or attention. Indeed, we might as well treat such questions as if their answers were formally independent of set theory, as for all we know they are (a possibility discussed further in Section 3.1).
238. u238 Response: One of the main purposes of this survey is to explain what we know now, relevant to the P ?= NP problem, that we didn’t know 10 or 20 or 30 years ago. It’s true that, if “progress” entails having a solution already in sight, or being able to estimate the time to a solution, I know of no progress of that kind! But by the same standard, one would have to say there was no “progress” toward Fermat’s Last Theorem in 1900—even as mathematicians, partly motivated by Fermat’s problem, were laying foundations of algebraic number theory that did eventually lead to Wiles’s proof. In this survey, I’ll try to convey how, over the last few decades, insights about circuit lower bounds, relativization and arithmetization, pseudorandomness and natural proofs, the “duality” between lower bounds and algorithms, the permanent and determinant manifolds, and more have transformed our understanding of what a P̸ = NP proof could look like.
239. u239 2Because I was asked: “yellow books” are the Springer mathematics books that line many mathematicians’ offices.
240. u240 I should point out that, even supposing P ?= NP is never solved, it’s already been remarkably fruitful as an “aspirational” or “flagship” question, helping to shape research in algorithms, cryptography, learning theory, derandomization, quantum computing, and other areas that theoretical computer scientists work on.
241. u241 Furthermore, later we’ll see examples of how progress in some of those other areas unexpectedly ended up tying back to the quest to prove P̸ = NP.
242. u242 1.2.6
243. u243 The Obviousness Objection Objection: It’s intuitively obvious that P̸ = NP. For that reason, a proof of P̸ = NP—confirming that indeed, we can’t do something that no reasonable person would ever have imagined we could do—gives almost no useful information.
244. u244 Response: This objection is perhaps less common among mathematicians than others, since were it upheld, it would generalize to almost all of mathematics! Like with most famous unsolved math problems, the quest to prove P̸ = NP is “less about the destination than the journey”: there might or might not be surprises in the answer itself, but there will certainly be huge surprises (indeed, there have already been huge surprises) along the way.
245. u245 More concretely: to make a sweeping statement like P̸ = NP, about what polynomial-time algorithms can’t do, will require an unprecedented understanding of what they can do. This will almost certainly entail the discovery of many new polynomial-time algorithms, some of which could have practical relevance. In Section 6, we’ll see much more subtle examples of the “duality” between algorithms and impossibility proofs, with progress on each informing the other.
246. u246 Of course, to whatever extent you regard P = NP as a live possibility, the Obviousness Objection isn’t open to you.
247. u247 1.2.7
248. u248 The Constructivity Objection Objection: Even if P = NP, the proof could be nonconstructive—in which case it wouldn’t have any of the amazing implications discussed in Section 1.1, because we wouldn’t know the algorithm.
249. u249 Response: A nonconstructive proof that an algorithm exists is indeed a theoretical possibility, though one that’s reared its head only a few times in the history of computer science.3 Even then, however, once we knew that an algorithm existed, we’d have a massive inducement to try to find
250. u250 The same is true if, for example, the first proof of P = NP only gave an n1000 algorithm, but it.
251. u251 we suspected that an n2 algorithm existed.4
252. u252 3The most celebrated examples of nonconstructive proofs that algorithms exist all come from the Robertson- Seymour graph minors theory, one of the great achievements of 20th-century combinatorics (for an accessible introduction, see for example Fellows [87]). The Robertson-Seymour theory typically deals with parameterized problems: for example, “given a graph G, decide whether G can be embedded on a sphere with k handles.”
253. u253 In those cases, typically a fast algorithm Ak can be abstractly shown to exist for every value of k. The central problem is that each Ak requires hard-coded data—in the above example, a finite list of obstructions to the desired embedding—that no one knows how to find given k, and whose size might also grow astronomically as a function of k. On the other hand, once the finite ( obstruction set for a given k was known, one could then use it to solve the problem for any graph G in time O
254. u254 , where the constant hidden by the big-O depended on k.
255. u255 Robertson-Seymour theory also provides a few examples of non-parameterized ( problems ( that are abstractly proved to be in P but with no bound on the exponent, or abstractly proved to be O
256. u256 but with no bound on the constant.
257. u257 Thus, one can’t rule out the possibility that the same would happen with an NP-complete problem, and Donald Knuth [155] has explicitly speculated that P = NP will be proven in that way.
258. u258 To me, however, it’s unclear whether he speculates this because there’s a positive reason for thinking it true, or just because it would be cool and interesting if it were true.
259. u259 4As an amusing side note, there’s a trick called Levin’s universal search [168], in which one “dovetails” over all
260. u260 1.3
261. u261 Further Reading
262. u262 ?= NP: Michael Sipser’s 1992 There were at least four previous major survey articles about P “The History and Status of the P versus NP Question” [241]; Stephen Cook’s 2000 “The P versus NP Problem” [74], which was written for the announcement of the Clay Millennium Prize; Avi Wigderson’s 2006 “P, NP, and Mathematics—A Computational Complexity Perspective” [269]; and Eric Allender’s 2009 “A Status Report on the P versus NP Question” [22].
263. u263 All four are excellent, so it’s only with trepidation that I add another entry to the crowded arena.
264. u264 I hope that, if nothing else, this survey shows how much has continued to occur through 2017.
265. u265 I cover several major topics that either didn’t exist a decade ago, or existed only in much more rudimentary form: for example, the algebrization barrier, “ironic complexity theory” (including Ryan Williams’s NEXP̸ ⊂ACC result), the “chasm at depth three” for the permanent, and the Mulmuley-Sohoni Geometric Complexity Theory program.
266. u266 The seminal papers that set up the intellectual framework for P ?= NP, posed it, and demonstrated its importance include those of Edmonds [84], Rabin [212], Cobham [73], Cook [76], Karp [145], and Levin [168].5
267. u267 See also Trakhtenbrot [259] for a survey of Soviet thought about perebor, as brute-force search was referred to in Russian in the 1950s and 60s.
268. u268 The classic text that introduced the wider world to P, NP, and NP-completeness, and that gave a canonical (and still-useful) list of hundreds of NP-complete problems, is Garey and Johnson [104]. Some recommended computational complexity theory textbooks—in rough order from earliest to most recent, in the material they cover—are Sipser [242], Papadimitriou [205], Sch¨oning [233], Moore and Mertens [184], and Arora and Barak [29]. Surveys on particular aspects of complexity theory will be recommended where relevant throughout the survey.
269. u269 Those seeking a nontechnical introduction to P ?= NP might enjoy Lance Fortnow’s charming book The Golden Ticket [94], or his 2009 popular article for Communications of the ACM [93]. My own Quantum Computing Since Democritus [6] gives something between a popular and a technical treatment.
270. u270 ?= NP and Central Related Concepts 2
271. u271 Formalizing P
272. u272 ?= NP problem is normally phrased in terms of Turing machines: a theoretical model of The P computation proposed by Alan Turing in 1936, which involves a one-dimensional tape divided into discrete squares, and a finite control that moves back and forth on the tape, reading and writing symbols. For a formal definition, see, e.g., Sipser [242] or Cook [74].
273. u273 In this survey, I won’t define Turing machines, for the simple reason that if you know any programming language—C, Java, Python, etc.—then you already know something that’s equivalent to Turing machines M1, M2, . . . (that is, for all t, runs M1, . . . , Mt for t steps each), halting when and if any Mi outputs a valid solution to one’s NP search problem. If we know P = NP, then we know this particular algorithm will find a valid solution, whenever one exists, in polynomial time—because clearly some Mi does so, and all the machines other than Mi increase the total running time by “only” a polynomial factor! With more work, one can even decrease this to a constant factor.
274. u274 Admittedly, however, the polynomial or constant factor will be so enormous as to negate this algorithm’s practical use.
275. u275 5As this survey was being completed, Bruce Kapron [75] posted on the Internet mimeographed notes, from a course taught by Stephen Cook at Berkeley in 1967, in which P̸ = NP is explicitly conjectured.
276. u276 This might be the first time the conjecture, in the precise form we know it today (up to the names of the classes), was made in print, for some definition of “in print.”
277. u277 Turing machines for our purposes. More precisely, the Church-Turing Thesis holds that virtually any model of computation one can define will be equivalent to Turing machines, in the sense that Turing machines can simulate that model and vice versa.
278. u278 A modern refinement, the Extended Church-Turing Thesis, says that moreover, these simulations will incur at most a polynomial overhead in time and memory.
279. u279 Nowadays, most computer scientists and physicists conjecture that quantum computation provides a counterexample to the Extended Church-Turing Thesis—possibly the only counterexample that can be physically realized. In Section 5.5, I’ll say a bit about how this changes the story.
280. u280 It’s also conceivable that access to a true random-number generator would let us violate the Extended Church-Turing Thesis, although most computer scientists conjecture that it doesn’t, for reasons that I’ll explain in Section 5.4.1.
281. u281 On the other hand, as long as we’re talking only about classical, digital, deterministic computation, the Extended Church-Turing Thesis remains on extremely solid ground.
282. u282 If we accept this, then there’s a well-defined notion of “solvable in polynomial time,” which is independent of the low-level details of the computer’s architecture: the instruction set, the specific rules for accessing memory, etc. This licenses us to ignore those details. The main caveat here is that there must be no a-priori limit on how much memory the computer can address, even though any program that runs for finite time will only address a finite amount of memory.6,7
283. u283 We can now define P and NP, in terms of Turing machines for concreteness—but, because of the Extended Church-Turing Thesis, the reader is free to substitute other computing formalisms such as Lisp programs, λ-calculus, stylized assembly language, or cellular automata.
284. u284 A language—the term is historical, coming from when theoretical computer science was closely connected to linguistics—just means a set of binary strings, L ⊆{0, 1}∗, where {0, 1}∗is the set of all binary strings of all (finite) lengths.
285. u285 Of course a language can be infinite, even though every string in the language is finite.
286. u286 One example is the language consisting of all palindromes: for instance, 00, 11, 0110, 11011, etc., but not 001 or 1100. A more interesting example is the language consisting of all binary encodings of prime numbers: for instance, 10, 11, 101, and 111, but not 100.
287. u287 A binary string x ∈{0, 1}∗, for which we want to know whether x ∈L, is called an instance of the general problem of deciding membership in L. Given a Turing machine M and an instance x, we let M (x) denote M run on input x (say, on a tape initialized to · · · 0#x#0 · · · , or x surrounded by delimiters and blank or 0 symbols). We say that M (x) accepts if it eventually halts and enters an “accept” state, and we say that M decides the language L if for all x ∈{0, 1}∗,
288. u288 The machine M may also contain a “reject” state, which M enters to signify that it has halted without accepting.
289. u289 Let |x| be the length of x (i.e., the number of bits).
290. u290 Then we say M is polynomial-time if there exists a polynomial p such that M (x) halts, either accepting or rejecting, after at most p (|x|) steps, for all x ∈{0, 1}∗.
291. u291 6The reason for this caveat is that, if a programming language were inherently limited to (say) 64K of memory, there would be only finitely many possible program behaviors, so in principle we could just cache everything in a giant lookup table.
292. u292 Many programming languages do impose a finite upper bound on the addressable memory, but they could easily be generalized to remove this restriction (or one could consider programs that store information on external I/O devices).
293. u293 7I should stress that, once we specify which computational models we have in mind—Turing machines, Intel machine code, etc.—the polynomial-time equivalence of those models is typically a theorem, though a rather tedious one.
294. u294 The “thesis” of the Extended Church-Turing Thesis, the part not susceptible to proof, is that all other “reasonable” models of digital computation will also be equivalent to those models.
295. u295 Now P, or Polynomial-Time, is the class of all languages L for which there exists a Turing machine M that decides L in polynomial time.
296. u296 Also, NP, or Nondeterministic Polynomial-Time, is the class of languages L for which there exists a polynomial-time Turing machine M, and a polynomial p, such that for all x ∈{0, 1}∗,
297. u297 x ∈L ⇐⇒∃w ∈{0, 1}p(|x|) M (x, w) accepts.
298. u298 In other words, NP is the class of languages L for which, whenever x ∈L, there exists a polynomialsize “witness string” w, which enables a polynomial-time “verifier” M to recognize that indeed x ∈L. Conversely, whenever x̸ ∈L, there must be no w that causes M (x, w) to accept.
299. u299 There’s an earlier definition of NP, which explains its ungainly name. Namely, we can define a nondeterministic Turing machine as a Turing machine that “when it sees a fork in the road, takes it”: that is, that’s allowed to transition from a single state at time t to multiple possible states at time t + 1.
300. u300 We say that a machine “accepts” its input x, if there exists a list of valid transitions between states, s1 →s2 →s3 →· · · , that the machine could make on input x that terminates in an accepting state sAccept. The machine “rejects” if there’s no such accepting path. The “running time” of such a machine is the maximum number of steps taken along any path, until the machine either accepts or rejects.8
301. u301 We can then define NP as the class of all languages L for which there exists a nondeterministic Turing machine that decides L in polynomial time. It’s clear that NP, so defined, is equivalent to the more intuitive verifier definition that we gave earlier. In one direction, if we have a polynomial-time verifier M, then a nondeterministic Turing machine can create paths corresponding to all possible witness strings w, and accept if and only if there exists a w such that
302. u302 In the other direction, if we have a nondeterministic Turing machine M′, then M (x, w) accepts.
303. u303 a verifier can take as its witness string w a description of a claimed path that causes M′ (x) to accept, then check that the path indeed does so.
304. u304 Clearly P ⊆NP, since an NP verifier M can just ignore its witness w, and try to decide in polynomial time whether x ∈L itself. The central conjecture is that this containment is strict. Conjecture 1 P̸ = NP.
305. u305 2.1
306. u306 NP-Completeness
307. u307 ?= NP but central to any discussion of it, is A further concept, not part of the statement of P NP-completeness.
308. u308 To explain this requires a few more definitions.
309. u309 An oracle Turing machine is a Turing machine that, at any time, can submit an instance x to an “oracle”: a device that, in a single time step, returns a bit indicating whether x belongs to some given language L. Though it sounds fanciful, this notion is what lets us relate different computational problems to each other, and as such is one of the central concepts in computer science. An oracle that answers all queries consistently with L is called an L-oracle, and we write ML to denote the (oracle) Turing machine
310. u310 We can then define PL, or P relative to L, as the class of all languages L′ for M with L-oracle.
311. u311 which there exists an oracle machine M such that ML decides L′ in polynomial time. If L′ ∈PL, then we also write L′ ≤T P
312. u312 P L, which means “L′ is polynomial-time Turing-reducible to L.”
313. u313 Note that polynomial-time Turing-reducibility is indeed a partial order relation (i.e., it’s transitive and reflexive).
314. u314 8Or one could consider the minimum number of steps along any accepting path; the resulting class will be the same.
315. u315 Figure 1: P, NP, NP-hard, and NP-complete
316. u316 A language L is NP-hard (technically, NP-hard under Turing reductions9) if NP ⊆PL.
317. u317 Informally, NP-hard means “at least as hard as any NP problem, under partial ordering by reductions.” That is, if we had a black box for an NP-hard problem, we could use it to solve all NP problems in polynomial time.
318. u318 Also, L is NP-complete if L is NP-hard and L ∈NP.
319. u319 Informally, NP-complete problems are the hardest problems in NP, in the sense that an efficient algorithm for any of them would yield efficient algorithms for all NP problems. (See Figure 2.1.)
320. u320 A priori, it’s not completely obvious that NP-hard or NP-complete problems even exist.
321. u321 The great discovery of theoretical computer science in the 1970s was that hundreds of problems of practical importance fall into these classes—giving order to what had previously looked like a random assortment of incomparable hard problems.
322. u322 Indeed, among the real-world NP problems that aren’t known to be in P, the great majority (though not all of them) are known to be NPcomplete.
323. u323 More concretely, consider the following languages:
324. u324 3Sat is the language consisting of all Boolean formulas φ over n variables, which consist of
325. u325 ANDs of “3-clauses” (i.e., ORs of up to 3 variables or their negations), such that there exists
326. u326 at least one assignment that satisfies φ. (Or strictly speaking, all encodings of such formulas
327. u327 as binary strings, under some fixed encoding scheme whose details don’t normally matter.)
328. u328 Here’s an example, for which one can check that there’s no satisfying assignment:
329. u329 (x ∨y ∨z) ∧(x ∨y ∨z) ∧(x ∨y) ∧(x ∨y) ∧(y ∨z) ∧(y ∨z)
330. u330 9In practice, often one only needs a special kind of Turing reduction called a many-one reduction or Karp reduction, which is a polynomial-time algorithm that maps every yes-instance of L′ to a yes-instance of L, and every no-instance of L′ to a no-instance of L. The additional power of Turing reductions—to make multiple queries to the L-oracle (with later queries depending on the outcomes of earlier ones), post-process the results of those queries, etc.—is needed only in a minority of cases.
331. u331 For that reason, most sources define NP-hardness in terms of many-one reductions. Nevertheless, for conceptual simplicity, throughout this survey I’ll talk in terms of Turing reductions.
332. u332 This translates to: at least one of x, y, z is true, at least one of x, y, z is false, x = y, and
333. u333 HamiltonCycle is the language consisting of all undirected graphs, for which there exists a
334. u334 cycle that visits each vertex exactly once: that is, a Hamilton cycle.
335. u335 (Again, here we mean
336. u336 all encodings of graphs as bit strings, under some fixed encoding scheme.
337. u337 For a string to
338. u338 belong to HamiltonCycle, it must be a valid encoding of some graph, and that graph must
339. u339 TSP (Traveling Salesperson Problem) is the language consisting of all encodings of ordered
340. u340 pairs ⟨G, k⟩, such that G is a graph with positive integer weights, k is a positive integer, and
341. u341 G has a Hamilton cycle of total weight at most k.
342. u342 Clique is the language consisting of all encodings of undirected graphs G, and positive
343. u343 integers k, such that G contains a clique (i.e., a subset of vertices all connected to each other)
344. u344 SubsetSum is the language consisting of all encodings of positive integer tuples ⟨a1, . . . , ak, b⟩,
345. u345 for which there exists a subset of the ai’s that sums to b.
346. u346 Col is the language consisting of all encodings of undirected graphs G, and positive integers
347. u347 k, such that G is k-colorable (that is, each vertex of G can be colored one of k colors, so that
348. u348 no two adjacent vertices are colored the same).
349. u349 All of these languages are easily seen to be in NP: for example, 3Sat is in NP because we can simply give a satisfying assignment to φ as a yes-witness, HamiltonCycle is in NP because we can give the Hamilton cycle, etc.
350. u350 The famous Cook-Levin Theorem says that one of these problems—3Sat—is also NP-hard, and hence NP-complete.
351. u351 Theorem 2 (Cook-Levin Theorem [76, 168]) 3Sat is NP-complete.
352. u352 A proof of Theorem 2 can be found in any theory of computing textbook (for example, [242]). Here I’ll confine myself to saying that Theorem 2 can be proved in three steps, each of them routine from today’s standpoint:
353. u353 (1) One constructs an artificial language that’s “NP-complete essentially by definition”: for ex-
354. u354 : ∃w ∈{0, 1}s such that M (x, w) accepts in ≤t steps
355. u355 where ⟨M⟩is a description of the Turing machine M. (Here s and t are encoded in so-called
356. u356 “unary notation,” to prevent a polynomial-size input from corresponding to an exponentially-
357. u357 large witness string or exponential amount of time, and thereby keep L in NP.)
358. u358 (2) One then reduces L to the CircuitSat problem, where we’re given as input a description
359. u359 of a Boolean circuit C built of AND, OR, and NOT gates, and asked whether there exists
360. u360 an assignment x ∈{0, 1}n for the input bits such that C (x) = 1.
361. u361 To give this reduction, in
362. u362 turn, is more like electrical engineering than mathematics: given a Turing machine M, one
363. u363 simply builds up a Boolean logic circuit that simulates the action of M on the input (x, w)
364. u364 for t time steps, whose size is polynomial in the parameters |⟨M⟩|, |x|, s, and t, and which
365. u365 outputs 1 if and only if M ever enters its accept state.
366. u366 (3) Finally, one reduces CircuitSat to 3Sat, by creating a new variable for each gate in the
367. u367 Boolean circuit C, and then creating clauses to enforce that the variable for each gate G
368. u368 equals the AND, OR, or NOT (as appropriate) of the variables for G’s inputs. For example,
369. u369 one can express the constraint a ∧b = c by
370. u370 One then constrains the variable for the final output gate to be 1, yielding a 3Sat instance
371. u371 φ that is satisfiable if and only if the CircuitSat instance was (i.e., iffthere existed an x
372. u372 Note that the algorithms to reduce L to CircuitSat and CircuitSat to 3Sat—i.e., to convert M to C and C to φ—run in polynomial time (actually linear time), so we indeed preserve NP-
373. u373 Also, the reason for the 3 in 3Sat is simply that an AND or OR gate has one output hardness.
374. u374 bit and two input bits, so it relates three bits in total. The analogous 2Sat problem turns out to be in P.
375. u375 Once one knows that 3Sat is NP-complete, “the floodgates are open.”
376. u376 One can then prove that countless other NP problems are NP-complete by reducing 3Sat to them, and then reducing those problems to others, and so on. The first indication of how pervasive NP-completeness really was came from Karp [145] in 1972. He showed, among many other results: Theorem 3 (Karp [145]) HamiltonCycle, TSP, Clique, SubsetSum, and Col are all NPcomplete.
377. u377 Today, so many combinatorial search problems have been proven NP-complete that, whenever one encounters a new such problem, a useful rule of thumb is that it’s “NP-complete unless it has a good reason not to be”!
378. u378 Note that, if any NP-complete problem is in P, then all of them are, and P = NP (since every NP problem can first be reduced to the NP-complete one, and then solved in polynomial time). Conversely, if any NP-complete problem is not in P, then none of them are, and P̸ = NP.
379. u379 One application of NP-completeness is to reduce the number of logical quantifiers needed to state the P̸ = NP conjecture, and thereby make it intuitively easier to grasp.
380. u380 A Σk-sentence is a sentence with k quantifiers over integers (or objects that can be encoded as integers), beginning with an existential (∃) quantifier; a Πk-sentence also has k quantifiers, but begins with a universal (∀) quantifier.
381. u381 Let T be the set of all Turing machines, and let P be the set of all polynomials. Also, given a language L, let L (x) be the characteristic function of L: that is, L (x) = 1 if x ∈L and L (x) = 0 otherwise. Then a “na¨ıve” statement of P̸ = NP would be as a Σ3-sentence:
382. u382 ∃L ∈NP ∀M ∈T , p ∈P ∃x (M (x)̸ = L (x) ∨M (x) runs for > p (|x|) steps) . (Here, by quantifying over all languages in NP, we really mean quantifying over all verification algorithms that define such languages.) But once we know that 3Sat (for example) is NP-complete, we can state P̸ = NP as just a Π2-sentence:
383. u383 ∀M ∈T , p ∈P ∃x (M (x)̸ = 3Sat (x) ∨M (x) runs for > p (|x|) steps) . In words, we can pick any NP-complete problem we like; then P̸ = NP is equivalent to the statement that that problem is not in P.
384. u384 2.2
385. u385 Other Core Concepts
386. u386 ?= NP question, and will be referred to later A few more concepts give a fuller picture of the P in the survey.
387. u387 In this section, I’ll restrict myself to concepts that were explored in the 1970s,
388. u388 ?= NP itself was formulated, and that are covered alongside P
389. u389 ?= NP around the same time as P in undergraduate textbooks.
390. u390 Other important concepts, such as nonuniformity, randomness, and one-way functions, will be explained as needed in Section 5.
391. u391 2.2.1
392. u392 Search, Decision, and Optimization For technical convenience, P and NP are defined in terms of languages or “decision problems,” which have a single yes-or-no bit as the desired output (i.e., given an input x, is x ∈L?).
393. u393 To put practical problems into this decision format, typically we ask something like: does there exist a solution that satisfies the following list of constraints? But of course, in real life we don’t merely want to know whether a solution exists; we want to find a solution whenever there is one!
394. u394 And given the many examples in mathematics where explicitly finding an object is harder than proving its existence, one might worry that this would also occur here.
395. u395 Fortunately, though, shifting our focus from decision problems to search problems doesn’t change the P ?= NP question at all, because of the following classic observation.
396. u396 Proposition 4 If P = NP, then for every language L ∈NP (defined by a verifier M), there’s a polynomial-time algorithm that, for all x ∈L, actually finds a witness w ∈{0, 1}p(n) such that M (x, w) accepts.
397. u397 Proof. The idea is to learn the bits of an accepting witness w = w1 · · · wp(n) one by one, by asking a series of NP decision questions. For example:
398. u398 Does there exist a w such that M (x, w) accepts and w1 = 0?
399. u399 Does there exist a w such that M (x, w) accepts, w1 = 0, and w2 = 0?
400. u400 Does there exist a w such that M (x, w) accepts, w1 = 1, and w2 = 0?
401. u401 Continue in this manner until all p (n) bits of w have been set.
402. u402 (This can also be seen as a binary search on the set of 2p(n) possible witnesses.)
403. u403 Note that there are problems for which finding a solution is believed to be much harder than deciding whether one exists.
404. u404 A classic example, as it happens, is the problem of finding a Nash equilibrium of a matrix game. Here Nash’s theorem guarantees that an equilibrium always exists, but an important 2006 result of Daskalakis et al. [81] gave evidence that there’s no polynomial-time algorithm to find an equilibrium.10
405. u405 The upshot of Proposition 4 is just that search and decision are equivalent for the NP-complete problems.
406. u406 10Technically, Daskalakis et al. showed that the search problem of finding a Nash equilibrium is complete for a complexity class called PPAD.
407. u407 This could be loosely interpreted as saying that the problem is “as close to NP-hard as it could possibly be, subject to Nash’s theorem showing why the decision version is trivial.”
408. u408 In practice, perhaps even more common than search problems are optimization { problems, where we have some efficiently-computable cost function, say C : {0, 1}n →
409. u409 , and the goal is to find a solution x ∈{0, 1}n that minimizes C (x).
410. u410 Fortunately, we can always reduce optimization problems to search and decision problems, by simply asking to find a solution x such that C (x) ≤K, and doing a binary search to find the smallest K for which such an x still exists. So again, if P = NP then all NP optimization problems are solvable in polynomial time.
411. u411 On the other hand, it’s important to remember that, while “is there an x such that C (x) ≤K?” is an NP question, “does minx C (x) = K?” and “does x∗minimize C (x)?” are presumably not NP questions in general, because no single x is a witness to a yes-answer.
412. u412 More generally, the fact that decision, search, and optimization all hinge on the same P ?= NP question has meant that many people, including experts, freely abuse language by referring to search and optimization problems as “NP-complete.” Strictly they should call such problems NPhard—we defined NP-hardness for languages, but the concept can be generalized to search and optimization problems—while reserving “NP-complete” for suitable associated decision problems.
413. u413 2.2.2
414. u414 The Twilight Zone: Between P and NP-complete We say a language L is NP-intermediate if L ∈NP, but L is neither in P nor NP-complete. Based on experience, one might hope not only that P̸ = NP, but that there’d be a dichotomy, with all NP problems either in P or else NP-complete. However, a classic result by Ladner [160] rules that possibility out.
415. u415 Theorem 5 (Ladner [160]) If P̸ = NP, then there exist NP-intermediate languages.
416. u416 While Theorem 5 is theoretically important, the NP-intermediate problems that it yields are extremely artificial (requiring diagonalization to construct). On the other hand, as we’ll see, there are also problems of real-world importance—particularly in cryptography, algebra, and number theory—that are believed to be NP-intermediate, and a proof of P̸ = NP could leave the status of those problems open.
417. u417 (Of course, a proof of P = NP would mean there were no NP-intermediate problems, since every NP problem would then be both NP-complete and in P.)
418. u418 2.2.3
419. u419 coNP and the Polynomial Hierarchy Let L = {0, 1}∗\L be the complement of L: that is, the set of strings not in L. Then the complexity class
420. u420 consists of the complements of all languages in NP. Note that this is not the same as NP, the set of all non-NP languages! Rather, L ∈coNP means that whenever x /∈L, there’s a short proof of nonmembership that can be efficiently verified. If L ∈NP, then L ∈coNP and vice versa. Likewise, if L is NP-complete, then L is coNP-complete (that is, in coNP and NP-hard) and vice versa. So for example, along with the NP-complete satisfiability we have the coNP-complete unsatisfiability, along with the NP-complete HamiltonCycle we have the coNP-complete HamiltonCycle (consisting of all encodings of graphs that lack a Hamilton cycle), etc.
421. u421 A natural question is whether NP is closed under complement: that is, whether NP = coNP. If P = NP, then certainly P = coNP, and hence NP = coNP also.
422. u422 On the other hand, we could imagine a world where NP = coNP even though P̸ = NP.
423. u423 In that world, there would always be
424. u424 short proofs of unsatisfiability (or of the nonexistence of cliques, Hamilton cycles, etc.), but those proofs could be intractable to find.
425. u425 A generalization of the P̸ = NP conjecture says that this doesn’t happen: Conjecture 6 NP̸ = coNP.
426. u426 A further generalization of P, NP, and coNP is the polynomial hierarchy PH.
427. u427 Defined by analogy with the arithmetic hierarchy in computability theory, PH is an infinite sequence of classes whose zeroth level equals P, and whose kth level (for k ≥1) consists of all problems that are in PL or NPL or coNPL, for some language L in the (k −1)st level.
428. u428 0 = P, and
429. u429 for all k ≥1.11
430. u430 A more intuitive definition of PH is as the class of languages that are definable using a polynomial-time predicate with a constant number of alternating universal and existential quantifiers: for example, L ∈ΠP 2
431. u431 2 if and only if there exists a polynomial-time machine M and polynomial p such that for all x,
432. u432 x ∈L ⇐⇒∀w ∈{0, 1}p(|x|) ∃z ∈{0, 1}p(|x|) M (x, w, z) accepts. NP is then the special case with just one existential quantifier, over witness strings w.
433. u433 If P = NP, then the entire PH “recursively unwinds” down to P: for example,
434. u434 Moreover, one can show that if ΣP k
435. u435 k+1 for any k, then all the levels above the kth come “crashing down” to ΣP k
436. u436 So for example, if NP = coNP, then PH = NP = coNP as
437. u437 On the other hand, a k collapse at the kth level isn’t known to imply a collapse at any lower well.
438. u438 level.
439. u439 Thus, we get an infinite sequence of stronger and stronger conjectures: first P̸ = NP, then NP̸ = coNP, then ΣP 2̸
440. u440 2 , and so on. In the limit, we can conjecture the following: Conjecture 7 All the levels of PH are distinct—i.e., the infinite hierarchy is strict.
441. u441 This is a generalization of P̸ = NP that many computer scientists believe—an intuition being that it would seem strange for the hierarchy to collapse at, say, the 37th level, without collapsing all the way down to P or NP.
442. u442 Conjecture 7 has many important consequences that aren’t known to follow from P̸ = NP itself.
443. u443 It’s also interesting to consider NP ∩coNP, which is the class of languages that admit short, easily-checkable proofs for both membership and non-membership.
444. u444 Here’s yet another strengthening of the P̸ = NP conjecture: Conjecture 8 P̸ = NP ∩coNP.
445. u445 Of course, if NP = coNP, then the P ?= NP ∩coNP question becomes equivalent to the original P ?= NP question. But it’s conceivable that P = NP ∩coNP even if NP̸ = coNP.
446. u446 11In defining the kth level of the hierarchy, we could also have given oracles for ΠP
447. u447 k−1: it doesn’t matter. Note also that “an oracle for complexity class C” should be read as “an oracle for any C-complete language L.”
448. u448 2.2.4
449. u449 Factoring and Graph Isomorphism As an application of these concepts, let’s consider two NP languages that are believed to be NPintermediate (if they aren’t simply in P). First, Fac—a language variant of the factoring problem— consists of all ordered pairs of positive integers ⟨N, k⟩such that N has a nontrivial divisor at most k. Clearly a polynomial-time algorithm for Fac can be converted into a polynomial-time algorithm to output the prime factorization (by repeatedly doing binary search to peel offN’s smallest divisor), and vice versa. Second, GraphIso—that is, graph isomorphism—consists of all encodings of pairs of undirected graphs ⟨G, H⟩, such that G ∼= H.
450. u450 It’s easy to see to see that Fac and GraphIso are both in NP.
451. u451 More interestingly, Fac is actually in NP ∩coNP.
452. u452 For one can prove that ⟨N, k⟩/∈Fac by exhibiting the unique prime factorization of N, and showing that it only involves primes greater than k.12 But this has the striking consequence that factoring can’t be NP-complete unless NP = coNP. The reason is the following.
453. u453 Proposition 9 ([56]) If any NP ∩coNP language is NP-complete, then NP = coNP, and hence PH collapses to NP.
454. u454 Proof. Suppose L ∈NP ∩coNP. Then PL ⊆NP ∩coNP, since one can prove the validity of every
455. u455 So if NP ⊆PL, then answer to every query to the L-oracle (whether the answer is ‘yes’ or ‘no’).
456. u456 NP ⊆NP ∩coNP and hence NP = coNP.
457. u457 The best currently-known classical algorithm for Fac, the so-called number field sieve, is conjectured (and empirically observed) to factor an n-digit integer in 2O(n1/3 log2/3 n) time.
458. u458 See for example Pomerance [210].
459. u459 GraphIso is not quite known to be in NP ∩coNP.
460. u460 However, it’s known to be in NP ∩coAM, where coAM is a certain probabilistic generalization of coNP.
461. u461 Klivans and van Melkebeek [154] showed that, under a plausible assumption about pseudorandom generators, we would have coNP = coAM and hence GraphIso∈NP ∩coNP.
462. u462 Even with no assumptions, Boppana, H˚astad, Zachos [55] proved the following.
463. u463 Theorem 10 ([55]) If GraphIso (or any other language in coAM) is NP-complete, then PH collapses to ΣP 2
464. u464 12This requires one nontrivial result, that every prime number has a succinct certificate—or in other words, that primality testing is in NP [211]. Since 2002, it is even known that primality testing is in P [15].
465. u465 As this survey was being written, Babai [35] announced the following breakthrough result. Theorem 11 (Babai [35]) GraphIso is solvable in quasipolynomial (that is, nlogO(1) n) time.13
466. u466 √n log n).
467. u467 The best previously-known bound, due to Babai and Luks [36] from 1983, had been 2O( Of course, Theorem 11 gives even more dramatic evidence that GraphIso is not NP-complete: if it was, then all NP problems would be solvable in quasipolynomial time as well.
468. u468 Based on experience—namely, that known algorithms can solve GraphIso extremely quickly for almost any graphs we can generate in practice—some computer scientists have conjectured for decades that GraphIso is in P, and certainly Theorem 11 is consistent with that conviction, but there remain significant obstacles to proving it.14
469. u469 2.2.5
470. u470 Space Complexity PSPACE is the class of languages L decidable by a Turing machine that uses a polynomial number of bits of space or memory, with no restriction on the number of time steps. Certainly P ⊆PSPACE, since in t time steps, a serial algorithm can access at most t memory cells.
471. u471 More generally, it’s not hard to see that P ⊆NP ⊆PH ⊆PSPACE, since in an expression like ∀x∃y φ (x, y), a PSPACE machine can loop over all possible values for x and y, using exponential time but reusing the same memory for each x, y pair. However, none of these containments have been proved to be strict.
472. u472 The following conjecture—asserting that polynomial space is strictly stronger than polynomial time—is perhaps second only to P̸ = NP itself in notoriety.
473. u473 Conjecture 12 P̸ = PSPACE.
474. u474 If P̸ = NP, then certainly P̸ = PSPACE as well, but the converse isn’t known.
475. u475 Just like there are hundreds of important NP-complete problems, there are many important PSPACE-complete problems as well.
476. u476 As a striking example, deciding which player has the win from a given board position, in two-player games of perfect information such as chess, checkers, or Go (when suitably generalized to n × n boards), is almost always a PSPACE-complete problem:15 see for example [248].
477. u477 Note that there’s no obvious NP witness for (say) White having a win in chess, since one would need to specify White’s response to every possible line of play by Black. So even if P = NP, chess might still be hard: again, the relevant question is P ?= PSPACE.
478. u478 One can also define a nondeterministic variant of PSPACE, called NPSPACE. But a 1970 result called Savitch’s Theorem [231] shows that actually PSPACE = NPSPACE.16
479. u479 The reasons for this are extremely specific to space, and don’t seem to suggest any avenue to proving P = NP, the analogous statement for time.
480. u480 13This was Babai’s original claim; then, on January 4, 2017, he posted an announcement scaling back the running
481. u481 √log n), because of an error discovered by Harald Helfgott.
482. u482 time claim to 22 e
483. u483 On January 9, 2017, however, Babai posted another announcement reinstating the original running time claim.
484. u484 14In particular, group isomorphism—that is, the problem of deciding whether two groups of order n, given by their multiplication tables, are isomorphic—is clearly solvable in ∼nlog n time and clearly reducible to GraphIso, but no algorithm for it better than ∼nlog n is known.
485. u485 15A caveat here is that the game needs to end after a polynomial number of moves—as would presumably be imposed by the timers in tournament play!
486. u486 If, by contrast, we allow chess endgames that last for exp (n) moves, then the computational complexity of chess is known to jump from PSPACE up to EXP [98]. This has the interesting consequence that, with no time limit, optimal play in n × n chess provably requires exponential time: no complexity hypothesis is needed.
487. u487 16A further surprising result from 1987, called the Immerman-Szelepcs´enyi Theorem [131, 254], says that the
488. u488 2.2.6
489. u489 Counting Complexity Given an NP search problem, besides asking whether a solution exists, it’s also natural to ask how many solutions there are. To capture this, in 1979 Valiant [262] defined the class #P (pronounced “sharp-P”, not “hashtag-P”!)
490. u490 Formally, a function f : {0, 1}∗→N is in #P if and only if there’s a polynomial-time Turing machine M, and a polynomial p, such that for all x ∈{0, 1}∗,
491. u491 Note that, unlike P, NP, and so on, #P is not a class of languages (i.e., decision problems). However, there are two ways we can compare #P to language classes.
492. u492 The first is by considering P#P: that is, P with a #P oracle.
493. u493 We then have NP ⊆P#P ⊆ PSPACE, as well as the following highly non-obvious inclusion, called Toda’s Theorem. Theorem 13 (Toda [258]) PH ⊆P#P.
494. u494 The second way is by considering a complexity class called PP (Probabilistic Polynomial-Time). PP can be defined as the class of languages L ⊆{0, 1}∗for which there exist #P functions f and g such that for all inputs x ∈{0, 1}∗,
495. u495 Equivalently, PP is the class of languages L for which there exists a probabilistic polynomial-time algorithm that merely needs to guess whether x ∈L, for each input x, with some probability
496. u496 It’s not hard to see that NP ⊆PP ⊆P#P.
497. u497 greater than 1/2.
498. u498 More interestingly, one can use binary search to show that PPP = P#P, so in that sense PP is “almost as strong as #P.”
499. u499 In practice, for any known NP-complete problem (3Sat, Clique, SubsetSum, etc.), the counting version of that problem (denoted #3Sat, #Clique, #SubsetSum, etc.) is #P-complete. Indeed, it’s open whether there’s any NP-complete problem that violates this rule.
500. u500 However, the converse statement is false.
501. u501 For example, the problem of deciding whether a graph has a perfect matching—that is, a set of edges that touches each vertex exactly once—is in P, but Valiant [262] showed that counting the number of perfect matchings is #P-complete.
502. u502 The #P-complete problems are believed to be “genuinely much harder” than the NP-complete problems, in the sense that—in contrast to the situation with PH—even if P = NP we’d still have no idea how to prove P = P#P. On the other hand, we do have the following nontrivial result. Theorem 14 (Stockmeyer [247]) Suppose P = NP.
503. u503 Then in polynomial time, we could approximate any #P function to within a factor of 1 ± class of languages decidable by a nondeterministic machine in f (n) space is closed under complement, for every “reasonable” memory bound f (n).
504. u504 (By contrast, Savitch’s Theorem produces a quadratic blowup when simulating nondeterministic space by deterministic space, and it remains open whether that blowup can be removed.)
505. u505 This further illustrates how space complexity behaves differently than we expect time complexity to behave.
506. u506 2.2.7
507. u507 Beyond Polynomial Resources Of course, we can consider many other time and space bounds besides polynomial. Before entering into this, I should offer a brief digression on the use of asymptotic notation in theoretical computer science, since such notation will also be used later in the survey.
508. u508 f (n) is O (g (n)) if there exist nonnegative constants A, B such that f (n) ≤Ag (n) + B for
509. u509 all n (i.e., g is an asymptotic upper bound on f).
510. u510 f (n) is Ω(g (n)) if g (n) is O (f (n)) (i.e., g is an asymptotic lower bound on f).
511. u511 f (n) is Θ (g (n)) if f (n) is O (g (n)) and g (n) is O (f (n)) (i.e., f and g grow at the same
512. u512 f (n) is o (g (n)) if for all positive A, there exists a B such that f (n) ≤Ag (n) + B for all n
513. u513 (i.e., g is a strict asymptotic upper bound on f).17
514. u514 Now let TIME (f (n)) be the class of languages decidable in O (f (n)) time, let NTIME (f (n)) be the class decidable in nondeterministic O (f (n)) time—that is, with a witness of size O (f (n)) that’s verified in O (f (n)) ∪ time—and ( let SPACE (f ∪ (n)) be the ( class decidable in O ∪ (f (n)) space.18 (
515. u515 nk) that’s We can verified then write in O (f P = (n)) ∪
516. u516 . It’s also interesting to study the exponential versions of these classes:
517. u517 Note that by “exponential,” here we mean not just 2O(n), but 2p(n) for any polynomial p.
518. u518 Just like we have P ⊆NP ⊆NPNP ⊆· · · ⊆PSPACE, so we also have
519. u519 EXP ⊆NEXP ⊆NEXPNP ⊆· · · ⊆EXPSPACE.
520. u520 Along with P ⊆PSPACE (Section 2.2.5), there’s another fundamental relation between time and space classes: Proposition 15 PSPACE ⊆EXP.
521. u521 Proof.
522. u522 Consider a deterministic machine whose state can be fully described by p (n) bits of information (e.g., the contents of a polynomial-size Turing machine tape, plus a few extra bits for
523. u523 Clearly such a machine has at most 2p(n) possible the location and internal state of tape head).
524. u524 17If f (n) is O (g (n)) but not Θ (g (n)), that does not necessarily mean that f (n) is o (g (n)): for example, consider f (n) = 2n (n/2 −⌊n/2⌋) and g (n) = n. Of course, ( such examples ( don’t arise often in practice.
525. u525 18Unlike P or PSPACE, classes like TIME
526. u526 can be sensitive to whether we’re using Turing machines, RAM machines, or some other model of computation. Thus, I’ll specify which I’m talking about whenever it’s relevant.
527. u527 Thus, after 2p(n) steps, either the machine has halted, or else it’s entered an infinite loop states.
528. u528 and will never accept. So to decide whether the machine accepts, it suffices to simulate it for 2p(n) steps.
529. u529 More generally, we get an infinite interleaved hierarchy of deterministic, nondeterministic, and space classes:
530. u530 P ⊆NP ⊆PH ⊆PSPACE ⊆EXP ⊆NEXP ⊆EXPSPACE ⊆· · · There’s also a “higher-up” variant of the P̸ = NP conjecture, which not surprisingly is also open: Conjecture 16 EXP̸ = NEXP.
531. u531 We can at least prove a close relationship between the P ?= NP and EXP ?= NEXP questions, via a trick called padding or upward translation: Proposition 17 If P = NP, then EXP = NEXP.
532. u532 Proof. Let L ∈NEXP, and let its verifier run in 2p(n) time for some polynomial p. Then consider the language
533. u533 which consists of the inputs in L, but “padded out with an exponential number of trailing zeroes.” Then L′ ∈NP, since verifying that x ∈{0, 1}n is in L takes 2p(n) time, which is linear in n + 2p(n) (the length of x02p(|x|)).
534. u534 But this means that L ∈EXP, since given x ∈{0, 1}n (an input for L), we can simply pad x out with 2p(n) trailing zeroes ourselves, then run the algorithm for L′ that takes time polynomial in n + 2p(n).
535. u535 For the same reason, if P = PSPACE, then EXP = EXPSPACE. On the other hand, padding only works in one direction: as far as anyone knows today, we could have P̸ = NP even if EXP = NEXP.
536. u536 One last remark: just as we can scale PSPACE up to EXPSPACE and so on, we can also scale PSPACE down to LOGSPACE, which is the class of languages L decidable by a Turing machine that uses only O (log n) bits of read/write memory, in addition to a read-only memory that stores the n-bit input itself.
537. u537 We can also define a nondeterministic counterpart, NLOGSPACE.19
538. u538 We then have: Proposition 18 NLOGSPACE ⊆P.
539. u539 Proof.
540. u540 An NLOGSPACE machine has, say, k log2 n bits of read/write memory, and therefore 2k log2 n = nk possible configurations of that memory. The machine accepts a given input x, if and only if there exists a path from its initial configuration to an accepting configuration.
541. u541 But this is just the reachability problem for a directed graph with nk vertices. Reachability is well-known to be solvable in polynomial time, for example by using depth-first search.
542. u542 Each of these inclusions is believed to be strict—with the possible exception of the first, about which opinion is divided. To date, however, no one has even ruled out the possibility that LOGSPACE = NP.
543. u543 19In the literature, LOGSPACE and NLOGSPACE are often simply called L and NL respectively.
544. u544 To summarize, P ?= NP is just the tip of an iceberg; there seems to be an extremely rich structure both below and above the NP-complete problems.
545. u545 Until we can prove P̸ = NP, however, most of that structure will remain conjectural.
546. u546 ?= NP 3
547. u547 Beliefs About P Just as Hilbert’s question turned out to have a negative answer, so too in this case, most computer scientists conjecture that P̸ = NP: that there exist rapidly checkable problems that aren’t rapidly solvable, and for which brute-force search is close to the best we can do. This is not a unanimous opinion.
548. u548 At least one famous computer scientist, Donald Knuth [155], has professed a belief that P = NP, while another, Richard Lipton [175], professes agnosticism.
549. u549 Also, in a poll of mathematicians and theoretical computer scientists conducted by William Gasarch [105] in 2002, there were 61 respondents who said P̸ = NP, but also 9 who said P = NP. (In a followup poll that Gasarch [106] conducted in 2012, there were 126 respondents who said P̸ = NP, and again 9 who said P = NP.)
550. u550 Admittedly, it can be hard to tell whether declarations that P = NP are meant seriously, or are merely attempts to be contrarian. However, we can surely agree with Knuth and Lipton that we’re far from understanding the limits of efficient computation, and that there are further surprises in store.
551. u551 In this section, I’d like to explain why, despite our limited understanding, many of us feel roughly as confident about P̸ = NP as we do about (say) the Riemann Hypothesis, or other conjectures in math—not to mention empirical sciences—that most experts believe without proof.20
552. u552 The first point is that, when we ask whether P = NP, we’re not asking whether heuristic optimization methods (such as Sat-solvers) can sometimes do well in practice, or whether there are sometimes clever ways to avoid exponential search.
553. u553 If you believe, for example, that there’s any cryptographic one-way function—that is, any transformation of inputs x →f (x) that’s easy to compute but hard to invert (see Section 5.3.1)—then that’s enough for P̸ = NP.
554. u554 Such an f need not have any “nice” mathematical structure: it could simply be, say, the evolution function of some arbitrary cellular automaton, or a scrambling operation applied over and over 1000 times, to the point where all known cryptanalytic methods can do no better than to scratch the surface.
555. u555 Sometimes amateurs become convinced that they can prove P = NP because of insight into the structure of a particular NP-complete problem, such as 3Sat or SubsetSum.
556. u556 However, this is a bit like thinking that one can solve a math problem because the problem is written in German rather than French. In other words, it fails to grapple with the central insight of NP-completeness: that for the purposes of P ?= NP, all these problems are just re-encodings of one another.
557. u557 It’s sometimes claimed that, when we consider P ?= NP, there’s a “symmetry of ignorance”: yes, we have no idea how to solve NP-complete problems in polynomial time, but we also have no idea how to prove that impossible, and therefore anyone’s free to believe whatever they like.
558. u558 In my opinion, what breaks the symmetry is the immense, well-known difficulty of proving lower bounds. Simply put: even if we suppose P̸ = NP, I don’t believe there’s any great mystery about why a proof has remained elusive.
559. u559 A rigorous impossibility proof is often a tall order, and many times in history—e.g., with Fermat’s Last Theorem, the Kepler Conjecture, or the problem of squaring
560. u560 20I like to joke that, if computer scientists had been physicists, we’d simply have declared P̸ = NP to be an observed law of Nature, analogous to the laws of thermodynamics. A Nobel Prize would even be given for the discovery of that law. (And in the unlikely event that someone later proved P = NP, a second Nobel Prize would be awarded for the law’s overthrow.)
561. u561 the circle—such a proof was requested centuries before mathematical understanding had advanced to the point where it became a realistic possibility.
562. u562 And as we’ll see in Section 6, today we know something about the difficulty of proving even “baby” versions of P̸ = NP; about the barriers that have been overcome and the others that remain.
563. u563 By contrast, if P = NP, then there’s at least a puzzle about why the entire software industry, over half a century, has failed to uncover any promising leads for, say, a fast algorithm to invert arbitrary one-way functions (just the algorithm itself, not necessarily a proof of correctness). The puzzle is heightened when we realize that, in many real-world cases—such as linear programming, primality testing, and network routing—fast methods to handle a problem in practice did come decades before a full theoretical understanding of why the methods worked.
564. u564 Another reason to believe P̸ = NP comes from the hierarchy theorems, which we’ll meet in Section 6.1.
565. u565 Roughly speaking, these theorems imply that “most” pairs of complexity classes are unequal; the trouble, in most cases, is merely that we can’t prove this for specific pairs!
566. u566 For example, in the chain of complexity classes P ⊆NP ⊆PSPACE ⊆EXP, we know that P̸ = EXP, which implies that at least one of P̸ = NP, NP̸ = PSPACE, and PSPACE̸ = EXP must hold. So we might say: given the provable reality of a rich lattice of unequal complexity classes, one needs to offer a special argument if one thinks two classes collapse, but not necessarily if one thinks they’re different.
567. u567 To my mind, however, the strongest argument for P̸ = NP involves the thousands of problems that have been shown to be NP-complete, and the thousands of other problems that have been shown to be in P.
568. u568 If just one of these problems had turned out to be both NP-complete and in P, that would’ve immediately implied P = NP. Thus, we could argue, the P̸ = NP hypothesis has had thousands of chances to be “falsified by observation.” Yet somehow, in every case, the NPcompleteness reductions and the polynomial-time algorithms “miraculously” avoid meeting each other—a phenomenon that I once described as the “invisible fence” [7].
569. u569 This phenomenon becomes particularly striking when we consider approximation algorithms for NP-hard problems, which return not necessarily an optimal solution but a solution within some
570. u570 To illustrate, there’s a simple polynomial-time algorithm that, given a 3Sat factor of optimal.
571. u571 instance φ, finds an assignment that satisfies at least a 7/8 fraction of the clauses.21
572. u572 Conversely, in 1997 Johan H˚astad [125] proved the following striking result.
573. u573 Theorem 19 (H˚astad [125]) Suppose there’s a polynomial-time algorithm that, given as input a satisfiable 3Sat instance φ, outputs an assignment that satisfies at least a 7/8 + ε fraction of the clauses, where ε > 0 is any constant.
574. u574 Theorem 19 is one (strong) version of the PCP Theorem [31, 32], which is considered one of the crowning achievements of theoretical computer science. The PCP Theorem yields many other examples of “sharp NP-completeness thresholds,” where as we numerically adjust the required solution quality, an optimization problem undergoes a sudden “phase transition” from being in P to being NP-complete. Other times there’s a gap between the region of parameter space known to
575. u575 21Strictly speaking, this is for the variant of 3Sat in which every clause must have exactly 3 literals, rather than at most 3.
576. u576 Also note that, if we allow the use of randomness, then we can satisfy a 7/8 fraction of the clauses in expectation by just setting each of the n variables uniformly at random! This is because a clause with 3 literals has 23 −1 = 7 ways to be satisfied, and only one way to be unsatisfied. A deterministic polynomial-time algorithm that’s guaranteed to satisfy at least 7/8 of the clauses requires only a little more work.
577. u577 be in P and the region known to be NP-complete. One of the major aims of contemporary research is to close those gaps, for example by proving the so-called Unique Games Conjecture [151].
578. u578 We see a similar “invisible fence” phenomenon in Leslie Valiant’s program of “accidental algorithms” [263].
579. u579 The latter are polynomial-time algorithms, often for planar graph problems, that exist for certain parameter values but not for others, for reasons that are utterly opaque if one doesn’t understand the strange cancellations that the algorithms exploit. A prototypical result is the following: Theorem 20 (Valiant [263]) Let Planar3Sat be the special case of 3Sat in which the bipartite graph of clauses and variables (with an edge between a variable and a clause whenever one occurs in the other) is planar.
580. u580 Now consider the following problem: given an instance of Planar3Sat which is monotone (i.e., has no negations), and in which each variable occurs in exactly two clauses, count the number of satisfying assignments mod k. This problem is in P for k = 7, but is NP-hard under randomized reductions for k = 2.22
581. u581 Needless to say (because otherwise you would’ve heard!), in not one of these examples have the “P region” and the “NP-complete region” of parameter space been discovered to overlap. For example, in Theorem 20, the NP-hardness proof just happens to break down if we ask about the number of solutions mod 7, the case where an algorithm is known.
582. u582 If P = NP then this is, at the least, an unexplained coincidence. If P̸ = NP, on the other hand, then it makes perfect sense.
583. u583 3.1
584. u584 Independent of Set Theory?
585. u585 Since the 1970s, there’s been speculation that P̸ = NP might be independent (that is, neither provable nor disprovable) from the standard axiom systems for mathematics, such as Zermelo- Fraenkel set theory. To be clear, this would mean that either
586. u586 (1) a polynomial-time algorithm for NP-complete problems doesn’t exist, but we can never prove
587. u587 it (at least not in our usual formal systems), or else
588. u588 (2) a polynomial-time algorithm for NP-complete problems does exist, but either we can never
589. u589 prove that it works, or we can never prove that it halts in polynomial time.
590. u590 Since P̸ = NP is an arithmetical statement (a Π2-sentence), we can’t simply excise it from mathematics, as some would do with questions in transfinite set theory, like the Continuum Hypothesis (CH) or the Axiom of Choice (AC). At the end of the day, a polynomial-time algorithm for 3Sat either exists or it doesn’t! But that doesn’t imply that we can prove which.
591. u591 In 2003, I wrote a survey article [1] about whether P ?= NP is formally independent, which somehow never got around to offering any opinion about the likelihood of that eventuality!
592. u592 So for the record: I regard the independence of P = NP as unlikely, just as I do for the Riemann hypothesis, Goldbach’s conjecture, and other unsolved problems of “ordinary” mathematics.
593. u593 At the least, I’d say that the independence of P ?= NP has the status right now of a “free-floating speculation” with little or no support from past mathematical experience.
594. u594 22Indeed, a natural conjecture would be that the problem is NP-hard for all k̸ = 7, but this remains open (Valiant, personal communication).
595. u595 Note also that a problem A being “NP-hard under randomized reductions” means that all of NP is decidable by a polynomial-time randomized algorithm with an oracle for A.
596. u596 There have been celebrated independence results over the past century, but as far as I know they all fall into five classes, none of which would encompass the independence of P ?= NP from ZF set theory:
597. u597 (1) Independence of statements that are themselves about formal systems: for example, that
598. u598 assert their own unprovability in a given system, or the system’s consistency.
599. u599 This is the
600. u600 class produced by G¨odel’s incompleteness theorems.23
601. u601 (2) Independence of statements in transfinite set theory, such as CH and AC. Unlike “ordinary”
602. u602 mathematical statements—P̸ = NP, the Riemann hypothesis, etc.—the set-theoretic state-
603. u603 ments can’t be phrased in the language of elementary arithmetic; only questions about their
604. u604 provability from various axiom systems are arithmetical.
605. u605 For that reason, one can question
606. u606 whether CH, AC, and so on need to have definite truth-values at all, independent of the axiom
607. u607 system. In any case, the independence of set-theoretic principles seems different in kind, and
608. u608 less “threatening,” than the independence of arithmetical statements.24
609. u609 (3) Independence from ZFC (ZF plus the Axiom of Choice) of various statements about met-
610. u610 ric spaces, measure theory, and projective sets.
611. u611 The statements about projective sets can
612. u612 nevertheless be proven if one assumes the existence of a suitable large cardinal.
613. u613 In other
614. u614 cases, such as the so-called Borel determinacy theorem [179], the statement can be proven
615. u615 in ZFC, but one really does need close to the full power of ZFC [99].
616. u616 These statements,
617. u617 while different from those in class (2), are open to the same objection: namely, that they’re
618. u618 about uncountable sets, so their independence seems less “threatening” than that of purely
619. u619 (4) Independence from “weak” systems, which don’t encompass all accepted mathematical rea-
620. u620 Goodstein’s Theorem [110], and the non-losability of the Kirby-Paris hydra game
621. u621 [153], are two examples of interesting arithmetical statements that can be proved using small
622. u622 amounts of set theory (or ordinal induction), but not within Peano arithmetic.
623. u623 The cele-
624. u624 brated Robertson-Seymour Graph Minor Theorem (see [87]), while not strictly arithmetical
625. u625 (since it quantifies over infinite lists of graphs), is another example of an important result
626. u626 that can be proven using ordinal induction, but that provably can’t be proven using axioms
627. u627 of similar strength to Peano arithmetic.
628. u628 For more see Friedman, Robertson, and Seymour
629. u629 (5) Independence from ZFC of unusual combinatorial statements.
630. u630 Harvey Friedman [101] has
631. u631 produced striking examples of such statements, which he claims are “perfectly natural”—
632. u632 i.e., they would eventually appear in the normal development of mathematics.
633. u633 There is
634. u634 In any case, the relevance of Friedman’s statements to
635. u635 computational complexity is remote at present.
636. u636 23I put statements that assert, e.g., the algorithmic incompressibility of a sufficiently long bit string into this class as well.
637. u637 24Note also that, by arguments going back to G¨odel, and generalized by the so-called Shoenfield absoluteness theorem [236], if an arithmetical statement, such as P̸ = NP, is provable in ZF using the Axiom of Choice or the Continuum Hypothesis, then it’s also provable in ZF alone.
638. u638 Indeed, this is true more generally, if we replace CH or AC with any statement proven independent of ZF via the forcing method.
639. u639 Of course, it’s possible that P ?= NP is independent of ZF, but that the relative consistency of P = NP and P̸ = NP with ZF is itself independent of ZF, and so on ad infinitum!
640. u640 But we can at least say that, if P̸ = NP (or for that matter, the Riemann hypothesis, Goldbach’s conjecture, etc.) were proven independent of ZF, it would be an unprecedented development: the first example of an independence result that didn’t fall into one of the five classes above.25
641. u641 The proof of independence would also have to be unlike any known independence proof.
642. u642 The techniques used to prove statements such as Goodstein’s Theorem independent of Peano arithmetic, actually prove independence from the stronger theory PA +Π1: that is, Peano arithmetic plus the set of all true arithmetical Π1-sentences (sentences with a single universal quantifier and no existential quantifiers).
643. u643 However, as Ben-David and Halevi [43] noticed, if P̸ = NP could likewise be proven independent of PA +Π1, that would mean that no Π1-sentence of PA implying P̸ = NP could hold. As a consequence, they deduce, for example, that NP-complete problems would have to be solvable in nlog log log log n time, and even in nα(n) time (where α (n) is the inverse Ackermann function26), for infinitely many input lengths n. In that sense, we would “almost” have P = NP.
644. u644 As Section 6 will discuss, there are various formal barriers—including the relativization, algebrization, and natural proofs barriers—that explain why certain existing techniques can’t be powerful enough to prove P̸ = NP.
645. u645 These barriers can be interpreted as proofs that P̸ = NP is unprovable from certain systems of axioms: namely, axioms that capture the power of the techniques in question (relativizing, algebrizing, or naturalizing techniques) [30, 133, 222].27
646. u646 In all these cases, however, the axiom systems are known not to capture all techniques in complexity theory: there are existing results that go beyond them.
647. u647 Thus, these barriers indicate weaknesses in certain techniques, rather than in the foundations of mathematics.
648. u648 25If a Π1-sentence like the Goldbach Conjecture or the Riemann Hypothesis is independent of ZF, then it’s true. This is because if it’s false, then it has a refutation in ZF (and much less) via any counterexample.
649. u649 However, this line of reasoning doesn’t apply to Π2-sentences like P̸ = NP.
650. u650 A counterexample to the Π2-sentence only provides a true Π1-sentence, one which might not be provable in ZF.
651. u651 26The Ackermann function, A (n), can be defined as f (n, n), where f (n, k) = f (n −1, f (n, k −1)) with boundary conditions f (n, 0) = f (n −1, 1) and f (0, k) = k + 1.
652. u652 This is one of the fastest-growing functions encountered in mathematics.
653. u653 Meanwhile, the inverse Ackermann function, α (n), is a monotone nondecreasing function with α (A (n)) = n: hence, one of the slowest-growing functions one encounters.
654. u654 More generally, Ben-David and Halevi [43] showed that if P̸ = NP is unprovable in PA +Π1, then NP-complete problems are solvable in nf−1(n) time, where f (n) is any function in the “Wainer hierarchy”: a hierarchy of fastgrowing functions, containing the Ackermann function, that can be proved to be total in PA.
655. u655 By contrast, in 1969 McCreight and Meyer [180] proved a theorem one of whose striking corollaries is the following: there exists a single computable time bound g (for which they give the algorithm), such that P = NP if and only if 3Sat is solvable in g (n) time.
656. u656 Their bound g (n) is “just barely” superpolynomial, and is constructed via a diagonalization procedure.
657. u657 The McCreight-Meyer Theorem explains, in particular, why Ben-David and Halevi’s result doesn’t show that if P̸ = NP is unprovable in PA +Π1, then NP-complete problems are solvable in nβ(n) time for every unbounded
658. u658 ?= NP computable function β.
659. u659 Namely, if it showed that, then it would actually show that PA +Π1 decides the P problem.
660. u660 27This is literally true for the relativization and algebrization barriers. For natural proofs, one can use the barrier to argue that P̸ = NP is unprovable from certain axioms of “bounded arithmetic,” but as far as I know there’s no known axiom set that precisely captures the power of natural proofs.
661. u661 4
662. u662 Why Is Proving P̸ = NP Difficult?
663. u663 Let’s suppose that P̸ = NP. Then given the disarming simplicity of the statement, why is proving it so hard? As mentioned above, complexity theorists have identified three technical barriers, called relativization [38], natural proofs [223], and algebrization [10], that any proof of P̸ = NP will need to overcome.
664. u664 They’ve also shown that it’s possible to surmount each of these barriers, though there are few results that surmount all of them simultaneously.
665. u665 The barriers will be discussed alongside progress toward proving P̸ = NP in Section 6.
666. u666 However, we can also say something more conceptual, and possibly more illuminating, about the meta-question of why it’s so hard to prove hardness.
667. u667 In my view, the central reason why proving P̸ = NP is hard is simply that, in case after case, there are amazingly clever ways to avoid brute-force search, and the diversity of those ways rivals the diversity of mathematics itself.
668. u668 And even if, as I said in Section 3, there seems to be an “invisible fence” separating the NP-complete problems from the slight variants of those problems that are in P—still, almost any argument we can imagine for why the NP-complete problems are hard would, if it worked, also apply to the variants in P.
669. u669 To illustrate, we saw in Section 2.1 that 3Sat is NP-complete. We also saw that 2Sat, which is like 3Sat except with two variables per clause rather than three, is in P: indeed, 2Sat is solvable in linear time. Other variants of satisfiability that are in P include HornSat (where each clause is an OR of arbitrarily many non-negated variables and at most one negated variable), and XorSat (where each clause is a linear equation mod 2, such as x2 ⊕x7 ⊕x9 ≡1 (mod 2)).
670. u670 Likewise, even though it’s NP-complete to decide whether a given graph is 3-colorable, we can decide in linear time whether a graph is 2-colorable. Also, even though SubsetSum is NP-complete, we can easily decide whether there’s a subset of a1, . . . , ak summing to b in time that’s nearly linear in a1 + · · · + ak. In other words, if each ai is required to be encoded in “unary” notation (that is, as a list of ai ones) rather than in binary, then SubsetSum is in P.
671. u671 As a more interesting example, finding the maximum clique in a graph is NP-complete, as are finding the minimum vertex cover, the chromatic number, and so on.
672. u672 Yet in the 1960s, Edmonds [84] famously showed the following.
673. u673 Theorem 21 (Edmonds [84]) Given an undirected graph G, there’s a polynomial-time algorithm to find a maximum matching: that is, a largest possible set of edges no two of which share a vertex.
674. u674 To a casual observer, matching doesn’t look terribly different from the other graph optimization problems, but it is different.
675. u675 Or consider linear, semidefinite, and convex programming. These techniques yield hundreds of optimization problems that seem similar to known NP-complete problems, and yet are solvable in P.
676. u676 A few examples are finding maximum flows, finding equilibria of two-player zero-sum games, training linear classifiers, and optimizing over quantum states and unitary transformations.28
677. u677 28I won’t have much to say about linear programming (LP) or semidefinite programming (SDP) in this survey, so perhaps this is as good a place as any to mention that today, we know a great deal about the impossibility of solving NP-complete problems in polynomial time by formulating them as “natural” LPs. This story starts in 1987, with a preprint ( by Swart [253] that claimed to prove P = NP by reducing the Traveling Salesperson Problem to an LP with
678. u678 ( O
679. u679 variables and constraints. Swart’s preprint inspired a landmark paper by Yannakakis [283] (making it possibly the most productive failed P = NP proof in history!), in which Yannakakis showed that there is no “symmetric” LP with no(n) variables and constraints that has the “Traveling Salesperson Polytope” as its projection onto a subset of the variables.
680. u680 Yannakakis also showed that the polytope corresponding to
681. u681 We can also give examples of “shocking” algorithms for problems that are clearly in P.
682. u682 Most famously, the problem of multiplying two n×n matrices, C = AB, seems like it should “obviously” require ∼n3 steps: ∼n steps for each of the n2 entries ( of the product matrix C. But in 1968, Strassen [250] discovered an algorithm that takes only O
683. u683 steps. There’s since been a long sequence of further improvements, culminating in ( an O
684. u684 algorithm by Coppersmith ( and Winograd [79], and its recent improvements to O
685. u685 by Vassilevska Williams [266], with a minuscule further improvement by Le Gall [103]. Thus, letting ω be the matrix multiplication exponent (i.e., the least ω such that n×n matrices can be multiplied in nω+o(1) time), we know today that ω ∈[2, 2.373].
686. u686 Some computer scientists conjecture that ω = 2; but in any case, just like with attempts to prove P̸ = NP, an obvious obstruction to proving ω > 2 is that the proof had better not yield ω = 3, or even a “natural-looking” bound like ω ≥2.5.
687. u687 The scope of polynomial-time algorithms might seem like a trite observation, incommensurate with the challenge of explaining why it’s so hard to prove P̸ = NP.
688. u688 Yet we have evidence to the
689. u689 Over the decades, there have been hundreds of flawed proofs announced for P̸ = NP. contrary.
690. u690 The attempt that received the most attention thus far, including coverage in The New York Times and other major media outlets, was that of Deolalikar [83] in 2010.
691. u691 But in every such case that I’m aware of, the proof could ultimately be rejected on the ground that, if it worked, then it would also yield superpolynomial lower bounds for problems known to be in P.
692. u692 With some flawed P̸ = NP proofs, this is easy to see: for example, perhaps the author proves that 3Sat must take exponential time, by some argument that’s fearsome in technical details, but ultimately boils down to “there are 2n possible assignments to the variables, and clearly any algorithm must spend at least one step rejecting each of them.” A general-purpose refutation of such arguments is simply that, if they worked, then they’d work equally well for 2Sat.
693. u693 Alternatively, one could point out that, as we’ll see in Section 5.1, it’s known how to solve 3Sat in (4/3)n time. So a P̸ = NP proof had better not imply a Ω(2n) lower bound for 3Sat.
694. u694 In the case of Deolalikar’s P̸ = NP attempt [83], the details were more complicated, but the bottom line ended up being similar. Deolalikar appealed to certain statistical properties of the set of satisfying assignments of a random 3Sat instance.
695. u695 The claim was that, for reasons having to do with logical definability, those statistical properties precluded 3Sat from having a polynomialtime algorithm.
696. u696 During an intense online discussion, however, skeptics pointed out that random XorSat—which we previously mentioned as a satisfiability variant in P—gives rise to solution sets indistinguishable from those of random 3Sat, with respect to the properties Deolalikar was using: see for example [265].
697. u697 This implied that there must be one or more bugs in the proof, though it still left the task of finding them (which was done later).
698. u698 None of this means that proving P̸ = NP is impossible. A priori, it might also have been hard to the maximum matching problem has no symmetric LP of subexponential size, but the polytope for the minimum spanning tree problem does have a polynomial-size LP.
699. u699 In general, expressibility by such an LP is sufficient for a problem to be in P, but not necessary.
700. u700 Later, in 2012, Fiorini et al. [88] substantially improved Yannakakis’s result, getting rid of the symmetry requirement.
701. u701 There have since been other major results in this direction: in 2014, Rothvoß [227] showed that the perfect matching polytope requires exponentially-large LPs (again with no symmetry requirement), while in 2015, Lee, Raghavendra, and Steurer [167] extended many of these lower bounds from linear to semidefinite programs.
702. u702 Collectively, these results rule out one “natural” approach to proving P = NP: namely, to start from famous NPhard optimization problems like TSP, and then find a polynomial-size LP or SDP that projects onto the polytope whose extreme points are the valid solutions. Of course, we can’t yet rule out the possibility that LPs or SDPs could help prove P = NP in some more indirect way (or via some NP-hard problem other than the specific ones that were studied); ruling that out seems essentially tantamount to proving P̸ = NP itself.
703. u703 imagine a proof of the unsolvability of the halting problem, but of course we know that such a proof exists. As we’ll see in Section 6.1, a central difference between the two cases is that methods from logic—namely, diagonalization and self-reference—worked to prove the unsolvability of the halting problem, but there’s a precise sense in which these methods can’t work (at least not by themselves) to prove P̸ = NP. A related difference comes from the quantitative character of P̸ = NP: somehow, any proof will need to explain why polynomial-time algorithm for 3Sat is impossible, even though a (4/3)n algorithm actually exists. In some sense, this need to make quantitative distinctions—to say that, yes, brute-force search can be beaten, but only by this much for this problem and by that much for that one—puts a lower bound on the sophistication of any P̸ = NP proof.
704. u704 5
705. u705 Strengthenings of the P̸ = NP Conjecture I’ll now survey various strengthenings of the P̸ = NP conjecture, which are often needed for applications to cryptography, quantum computing, fine-grained complexity, and elsewhere. Some of these strengthenings will play a role when, in Section 6, we discuss the main approaches to proving P̸ = NP that have been tried.
706. u706 5.1
707. u707 Different Running Times There’s been a great deal of progress on beating brute-force search for many NP-complete problems, even if the resulting algorithms still take exponential time.
708. u708 For example, Sch¨oning proved the following in 1999.
709. u709 Theorem 22 (Sch¨oning [232]) There’s a randomized algorithm that solves 3Sat in O((4/3)n) time.
710. u710 For many NP-complete problems like HamiltonCycle, for which the obvious brute-force algorithm takes ∼n! time, it’s also possible to reduce the running time to O (2n), or sometimes even to O (cn) for c < 2, through clever tricks such as dynamic programming (discussed in Cormen et al. [80], or any other algorithms textbook).
711. u711 How far can these algorithms be pushed? For example, is it possible that 3Sat could be solved
712. u712 √n) time, as various NP problems like Factoring are known to be (see Section 2.2.4)? An in 2O( important conjecture called the Exponential Time Hypothesis, or ETH, asserts that the answer is no: Conjecture 23 (Exponential Time Hypothesis) Any deterministic algorithm for 3Sat takes Ω(cn) steps, for some constant c > 1.
713. u713 ETH is an ambitious strengthening of P̸ = NP.
714. u714 Even assuming P̸ = NP, there’s by no means a consensus in the field that ETH is true—let alone still further strengthenings, like the Strong Exponential Time Hypothesis or SETH, which asserts that any algorithm for kSat requires Ω((2 −ε)n) time, for some ε that goes to zero as k →∞. SETH, of course, goes even further out on a limb than ETH does, and some algorithms researchers have been actively working to disprove SETH (see [281] for example).
715. u715 One thing we do know, however, is that if ETH or SETH hold, there are numerous implications that are not known to follow from P̸ = NP alone.
716. u716 One example concerns the problem of approximating the value of a “two-prover free game”: here Impagliazzo, Moshkovitz, and I [9] gave an
717. u717 nO(log ( n)-time algorithm, but we also proved that any f (n)-time algorithm would imply a nearly
718. u718 ( f
719. u719 Thus, assuming ETH, our quasipolynomial-time algorithm is essentially optimal, and our reduction from 3Sat to free games is also essentially optimal.
720. u720 A second example comes from recent work of Backurs and Indyk [37], who studied the problem of EditDistance: that is, given two strings, computing the minimum number ( of insertions, deletions, and replacements needed to transform one string to the other. Here an O
721. u721 algorithm has long been known [268].
722. u722 In a 2015 breakthrough, Backurs and Indyk [37] showed that algorithm to be essentially optimal, assuming SETH.
723. u723 Even more recently, Abboud et al. [11] have shown that edit distance requires nearly quadratic time under a “safer” conjecture: Theorem 24 (Abboud et al. [11]) Suppose that the circuit satisfiability problem, for circuits of depth o (n), can’t be solved in (2 −ε)n time for any ε > 0.
724. u724 Then EditDistance requires n2−o(1) time.
725. u725 In any case, we currently have no idea how to make similarly “fine-grained” statements about running times assuming only P̸ = NP.
726. u726 5.2
727. u727 Nonuniform Algorithms and Circuits P ?= NP asks whether there’s a single algorithm that, for every input size n, solves an NP-complete problem like 3Sat in time polynomial in n. But we could also allow a different algorithm for each input size.
728. u728 For example, it often happens in practice that a na¨ıve algorithm works the fastest for inputs up to a certain size (say n = 100), then a slightly clever algorithm starts doing better, then at n ≥1000 a very clever algorithm starts to outperform the slightly clever algorithm, and so on. In such a case, we might not even know whether the sequence terminates with a “maximally clever algorithm,” or whether it goes on forever.29
729. u729 To capture these situations, let P/poly be the class of languages L for which there exists a polynomial-time Turing machine M, as well as an infinite set of “advice strings” a1, a2, . . ., where an is p (n) bits long for some polynomial p, such that for all n and all x ∈{0, 1}n, we have
730. u730 An equivalent way to define P/poly is as the class of languages recognized by a family of polynomialsize circuits, one for each input size n.
731. u731 In theoretical computer science, a circuit just means a directed acyclic30 graph Cn of Boolean logic gates (such as AND, OR, NOT), with the input bits x1, . . . , xn at the bottom, and an output bit determining whether x ∈L at the top. The size of a circuit is the number of gates in it. The fanin of a circuit is the maximum number of input wires
732. u732 29The so-called Blum speedup theorem [52] shows that we can artificially construct problems for which the sequence continues forever, there being no single fastest algorithm.
733. u733 No “natural” problem is known to have this behavior, though it’s possible that some do. There are some natural problems, such as NP ∩coNP and #P-complete problems, that are known not to have this behavior.
734. u734 The reason is that we can give an explicit algorithm for these problems that must be nearly asymptotically optimal, and that succeeds for all but finitely many input lengths n: namely, an algorithm that simulates the lexicographically-first log n Turing machines until one of them supplies a proof or interactive proof for the correct answer.
735. u735 30Despite the term “circuit,” which comes from electrical engineering, circuits in theoretical computer science are ironically free of cycles; they proceed from the inputs to the output via layers of logic gates.
736. u736 that can enter a gate g, while the fanout is the maximum number of output wires that can emerge from g (that is, the number of other gates that can depend directly on g’s output). For now, we’re considering circuits with a fanin of 2 and unlimited fanout.
737. u737 We call P/poly the nonuniform generalization of P, where ‘nonuniform’ just means that the circuit Cn could have a different structure for each n (i.e., there need not be an efficient algorithm that outputs a description of Cn given n as input).
738. u738 Certainly P ⊂P/poly, but there’s no containment in the other direction.31
739. u739 Now, the nonuniform version of the P̸ = NP conjecture is the following.
740. u740 Conjecture 25 NP̸ ⊂P/poly.
741. u741 If P = NP, then certainly NP ⊂P/poly, but the converse need not hold. About the closest we have to a converse is the Karp-Lipton Theorem [146]: Theorem 26 If NP ⊂P/poly, then PH collapses to ΣP 2 Proof. Consider a problem in ΠP 2
742. u742 2 : say, “for all x ∈{0, 1}p(n), does there exist a y ∈{0, 1}p(n) such that A (x, y) accepts?”, for some polynomial p and polynomial-time algorithm A.
743. u743 Assuming NP ⊂P/poly, we can solve that problem in ΣP 2
744. u744 “Does there exist a circuit C such that for all x, the algorithm A (x, C (x)) accepts?”
745. u745 For if NP ⊂P/poly and ∀x∃yA (x, y) is true, then clearly there exists a polynomial-size circuit C that takes x as input, and outputs a y such that A (x, y) accepts.
746. u746 So we can simply use the existential quantifier in our ΣP 2
747. u747 2 algorithm to guess a description of that circuit.
748. u748 We conclude that, if NP ⊂P/poly, then ΠP 2
749. u749 But this is known to cause a collapse of the entire polynomial hierarchy to ΣP 2
750. u750 In summary, while most complexity theorists conjecture that NP̸ ⊂P/poly, as far as we know it’s a stronger conjecture than P̸ = NP.
751. u751 Indeed, it’s even plausible that future techniques could prove P̸ = NP without proving NP̸ ⊂P/poly: for example, as we’ll discuss in Section 6.1, we can currently prove P̸ = EXP, but can’t currently prove EXP̸ ⊂P/poly, or even NEXP̸ ⊂P/poly. Despite this, as we’ll see in Section 6, most techniques that have been explored for proving P̸ = NP, would actually yield the stronger result NP̸ ⊂P/poly if they worked. For that reason, P/poly plays a central role in work on the P ?= NP question.
752. u752 There’s one other aspect of circuit complexity that will play a role later in this survey: depth. The depth of a circuit simply means the length of the longest path from an input bit to the output bit—or, if we think of the logic gates as organized into layers, then the number of layers. There’s a subclass of P/poly called NC1 (the NC stands for “Nick’s Class,” after Nick Pippenger), which consists of all languages that are decided by a family of circuits that have polynomial size and also depth O (log n).32 One can also think of NC1 as the class of problems solvable in logarithmic time
753. u753 31This is a rare instance where non-containment can actually be proved. For example, any unary language (i.e., language of the form {0n : n ∈S}) is clearly in P/poly, since the nth circuit can just hardwire whether n ∈S.
754. u754 But there’s an uncountable infinity of unary languages, whereas { P is countable, so almost all unary } languages are outside P.
755. u755 Alternatively, we can observe that the unary language
756. u756 can’t be in P, since it’s simply a version of the halting problem.
757. u757 32If each logic gate depends on at most 2 inputs, then log2 n is the smallest depth that allows the output to depend on all n input bits.
758. u758 It’s conjectured that P̸ ⊂NC1 (nonuniformly) using a polynomial number of parallel processors.
759. u759 (that is, not all efficient algorithms can be parallelized), but alas, even showing NEXP̸ ⊂NC1 remains open at present.
760. u760 Another way to define NC1 is as the class of languages decidable by a family of polynomial-size Boolean formulas. In theoretical computer science, a formula just means a circuit where every gate has fanout 1 (that is, where a gate cannot have its output fed as input to multiple other gates). To see the equivalence: in one direction, by replicating subcircuits wherever necessary, clearly any circuit of depth d and size s can be “unraveled” into a formula of depth d and size at most 2ds, which is still polynomial in n if d = O (log n) and s = nO(1).
761. u761 In the other direction, there’s an extremely useful fact proved by Spira [245], called “depth reduction.”
762. u762 Proposition 27 (Spira [245]) Given any Boolean formula of size S, there is an equivalent formula of size SO(1) and depth O (log S).33
763. u763 Because of Proposition 27, the minimum depth D of any formula for a Boolean function f is simply Θ (log S), where S is the minimum size of any formula for f. For circuits, by contrast, size and depth are two independent variables, which might in general be related only by D ≤S ≤2D.
764. u764 5.3
765. u765 Average-Case Complexity If P̸ = NP, that means that there are NP problems for which no Turing machine succeeds at solving all instances in polynomial time.
766. u766 But often, especially in cryptography, we need more than that. It would be laughable to advertise a cryptosystem on the grounds that there exist messages that are hard to decode!
767. u767 So it’s natural to ask whether there are NP problems that are hard “in the average case” or “on random instances,” rather than merely in the worst case.
768. u768 More pointedly, does the existence of such problems follow from P̸ = NP, or is it a different, stronger assumption?
769. u769 The first step is to clarify what we mean by a “random instance.”
770. u770 For some NP-complete problems, it makes sense to ask about a uniform random instance: for example, we can consider 3Sat with n variables and m = αn uniformly-random clauses (for some constant α), or 3Coloring on an Erd˝os-R´enyi random graph.34
771. u771 In those cases, the difficulty tends to vary wildly with the problem and the precise distribution.
772. u772 With 3Sat, for example, if the clause/variable ratio α is too small, then random instances are trivially satisfiable, while if α is too large, then they’re trivially unsatisfiable.
773. u773 But there’s a “sweet spot,” α ≈4.2667, where random 3Sat undergoes a phase transition from satisfiable to unsatisfiable, and where the difficulty seems to blow up accordingly. Even at the threshold, however, random 3Sat might still be much easier than worstcase 3Sat: the breakthrough survey propagation algorithm [57] can solve random 3Sat quickly, even for α extremely close to the threshold.35
774. u774 More generally, there’s been a great deal of work on understanding particular distributions over instances, often using tools from statistical physics: for an accessible introduction, see for example Moore and Mertens [184].
775. u775 Unfortunately, there are almost no known reductions among these sorts of distributional problems, which would let us say that if one of them is hard then so is another.
776. u776 The reason is that almost any imaginable reduction from problem A to problem B will map a random instance of A to an extremely special, non-random instance of B.
777. u777 33Bshouty, ( Cleve, and Eberly [59] showed that the size of the depth-reduced formula can even be taken to be
778. u778 ( O
779. u779 34That is, a graph where every two vertices are connected by an edge with independent probability p.
780. u780 35But making matters more complicated still, survey propagation fails badly on random 4Sat.
781. u781 This means that, if we want to pick random instances of NP-complete problems and be confident they’re hard, then we might need carefully-tailored distributions. Levin [169], and Li and Vit´anyi [170], observed that there exists a “universal distribution” D—independent of the specific problem— with the remarkable property that any algorithm that fails on any instance, will also fail with high probability with respect to instances drawn from D. Briefly, one constructs D by giving each string x ∈{0, 1}∗a probability proportional to 2−K(x), where K (x) is the Kolmogorov complexity of x: that is, the number of bits in the shortest computer program whose output is x. One then argues that, given any algorithm A, one can design a short computer program that brute-force searches for the first instances on which A fails—and for that reason, if there are any such instances, then D will assign them a high probability!
782. u782 In this construction, the catch is that there’s no feasible way actually to sample instances from the magical distribution D. Thus, given a family of distributions D = {Dn}n≥1, where Dn is over {0, 1}p(n) (for some polynomial p), call D efficiently samplable if there exists a Turing machine that takes as input a positive integer n and a uniformly random string r ∈{0, 1}q(n) (for some polynomial q), and that outputs a sample from Dn in time polynomial in n. Then the real question, we might say, is whether there exist NP-complete problems that are hard on average with respect to efficiently samplable distributions. More formally, does the following conjecture hold?
783. u783 Conjecture 28 (NP Hard on Average) There exists a language L ∈NP, as well as an efficiently samplable family of distributions D = {Dn}n≥1, such that for all polynomial-time algorithms A, there exists an n such that
784. u784 Here L (x) ∈{0, 1} denotes the characteristic function of L.
785. u785 Note that, if Conjecture 28 holds, then for every NP-hard language L′, there exists an efficiently samplable family of distributions D′ such that L′ is hard on average with respect to instances drawn
786. u786 We can obtain this D′ by simply starting with a sample from D, and then applying the from D′.
787. u787 reduction from L to L′.
788. u788 It’s a longstanding open problem whether P̸ = NP implies Conjecture 28.
789. u789 There are NP problems—one famous example being the discrete logarithm problem—that are known to have the remarkable property of worst-case/average-case equivalence.
790. u790 That is, any polynomial-time algorithm for these problems that works on (say) 10% of instances implies a polynomial-time algorithm for all instances; and conversely, if the problem is hard at all then it’s hard on average. However, despite decades of work, no one has been able to show worst-case/average-case equivalence for any NP-complete problem (with respect to any efficiently samplable distribution), and there are known obstacles to such a result.
791. u791 For details, see for example the survey by Bogdanov and Trevisan [53].
792. u792 5.3.1
793. u793 Cryptography and One-Way Functions One might hope that, even if we can’t base secure cryptography solely on the assumption that P̸ = NP, at least we could base it on Conjecture 28. But there’s one more obstacle. In cryptography, we don’t merely need NP problems for which it’s easy to generate hard instances: rather, we need NP problems for which it’s easy to generate hard instances, along with secret solutions to those instances. This motivates the definition of a one-way function (OWF), perhaps the central concept
794. u794 Let f = {fn}n≥1 be a family of functions, with fn : {0, 1}n →{0, 1}p(n) of modern cryptography.
795. u795 for some polynomial p. Then we call f a one-way function family if
796. u796 (1) fn is computable in time polynomial in n, but
797. u797 (2) fn is hard to invert: that is, for all polynomial-time algorithms A and polynomials q, we have
798. u798 We then make the following conjecture.
799. u799 Conjecture 29 There exists a one-way function family.
800. u800 Conjecture 29 is stronger than Conjecture 28, which in turn is stronger than P̸ = NP. Indeed, it’s not hard to show the following.
801. u801 Proposition 30 Conjecture 29 holds if and only if there exists a fast way to generate hard random 3Sat instances with “planted solutions”: that is, an efficiently samplable family of distributions D = {Dn}n over (φ, x) pairs, where φ is a satisfiable 3Sat instance and x is a satisfying assignment to φ, such that for all polynomial-time algorithms A and all polynomials q,
802. u802 φ∼Dn Pr [A (φ) finds a satisfying assignment to φ] < for all sufficiently large n.
803. u803 Proof. Given a one-way function family f, we can generate a hard random 3Sat instance with a planted solution by choosing x ∈{0, 1}n uniformly at random, computing fn (x), and then using the Cook-Levin Theorem (Theorem 2) to construct a 3Sat instance that encodes the problem of finding a preimage of fn (x).
804. u804 Conversely, given a polynomial-time algorithm that takes as input a positive integer n and a random string r ∈{0, 1}p(n) (for some polynomial p), and that outputs a hard 3Sat instance φr together with a planted solution xr to φr, the function fn (r) := φr will necessarily be one-way, since inverting fn would let us find a satisfying assignment to φr.
805. u805 Conjecture 29 turns out to suffice for building most of the ingredients of private-key cryptography, notably including pseudorandom generators [126] and pseudorandom functions [107]. Furthermore, while Conjecture 29 is formally stronger than P̸ = NP, Proposition 30 suggests that the two conjectures are conceptually similar: “all we’re asking for” is a hard NP problem, together with a fast way to generate hard solved instances of it!
806. u806 This contrasts with the situation for public-key cryptography—i.e., the kind of cryptography that doesn’t require any secrets to be shared in advance, and which is used for sending creditcard numbers over the web.
807. u807 To create a secure public-key cryptosystem, we need something even stronger than Conjecture 29: for example, a trapdoor OWF,36,37 which is an OWF with the
808. u808 36There are closely-related objects, such as “lossy” trapdoor OWFs (see [208]), that also suffice for building publickey cryptosystems.
809. u809 37By contrast, public-key digital signature schemes—that is, ways to authenticate a message without a shared secret key—can be constructed under the sole assumption that OWFs exist. See for example Rompel [225].
810. u810 additional property that it becomes easy to invert if we’re given a secret “trapdoor” string generated along with the function.
811. u811 We do, of course, have candidates for secure public-key cryptosystems, which are based on problems such as factoring, discrete logarithms (over both multiplicative groups and elliptic curves), and finding planted short nonzero vectors in lattices.
812. u812 To date, however, all public-key cryptosystems require “sticking our necks out,” and conjecturing the hardness of some specific NP problem, something with much more structure than any known NP-complete problem.
813. u813 In other words, for public-key cryptography, today one has to make conjectures that go fundamentally beyond P̸ = NP, or even the existence of OWFs.
814. u814 Even if someone proved P̸ = NP or Conjecture 29, reasonable doubt could still remain about the security of known public-key cryptosystems.
815. u815 5.4
816. u816 Randomized Algorithms Even assuming P̸ = NP, we can still ask whether NP-complete problems can be solved in polynomial time with help from random bits. This is a different question than whether NP is hard on average: whereas before we were asking about algorithms that solve most instances (with respect to some distribution), now we’re asking about algorithms that solve all instances, for most choices of some auxiliary random numbers.
817. u817 Historically, algorithm designers have often resorted to randomness, to deal with situations where most choices that an algorithm could make are fine, but any specific choice will lead to terrible behavior on certain inputs.
818. u818 For example, in Monte Carlo simulation, used throughout science and engineering, we estimate the volume of a high-dimensional object by just sampling random points, and then checking what fraction of them lie inside.
819. u819 Another example concerns primality testing: that is, deciding the language
820. u820 Primes = {N : N is a binary encoding of a prime number} .
821. u821 In modern cryptosystems such as RSA, generating a key requires choosing large random primes, so it’s just as important that primality testing be easy as that the related factoring problem be hard! In the 1970s, Rabin [213] and Solovay and Strassen [244] showed how to decide Primes in time polynomial in log N (i.e., the number of digits of N).
822. u822 The catch was that their algorithms were randomized: in addition to N, they required a second input r; and for each N, the algorithms were guaranteed to succeed for most r’s but not all of them. Miller [183] also proposed a deterministic polynomial-time algorithm for Primes, but could only prove the algorithm correct assuming the Extended Riemann Hypothesis.
823. u823 Finally, after decades of work on the problem, in 2002 Agrawal, Kayal, and Saxena [15] gave an unconditional proof that Primes is in P.
824. u824 In other words, if we only care about testing primality in polynomial time, and not about the degree of the polynomial, then randomness was never needed after all.
825. u825 A third example of the power of randomness comes from the polynomial identity testing (PIT) problem. Here we’re given as input a circuit or formula, composed of addition and multiplication gates, that computes a polynomial p : F →F over a finite field F. The question is whether p is the identically-zero polynomial—that is, whether the identity p (x) = 0 holds. If deg (p) ≪|F|, then algebra immediately suggests a way to solve this problem: simply pick an x ∈F uniformly at random and check whether p (x) = 0.
826. u826 Since a nonzero polynomial p can vanish on at most deg (p) points, the probability that we’ll “get unlucky” and choose one of those points is at most deg (p) / |F|. To
827. u827 this day, no one knows of any deterministic approach that achieves similar performance.38 Derandomizing PIT—that is, replacing the randomized algorithm by a comparably-efficient deterministic one—is considered one of the frontier problems of theoretical computer science.
828. u828 For details, see for example the survey of Shpilka and Yehudayoff[239].
829. u829 5.4.1
830. u830 BPP and Derandomization What’s the power of randomness more generally? Can every randomized algorithm be derandomized, as ultimately happened with Primes?
831. u831 To explore these issues, complexity theorists study several randomized generalizations of the class P. We’ll consider just one of them: Bounded-Error Probabilistic Polynomial-Time, or BPP, is the class of languages L ⊆{0, 1}∗for which there exists a polynomial-time Turing machine M, as well as a polynomial p, such that for all inputs x ∈{0, 1}n,
832. u832 In other words, for every x, the machine M must correctly decide whether x ∈L “most of the time” (that is, for most choices of r). Crucially, here we can easily replace the constant 2/3 by any other number between 1/2 and 1, or even by a function like 1 −2−n. So for example, if we wanted to know x ∈L with 0.999999 confidence, then we’d simply run M several times, with different independent values of r, and then output the majority vote among the results.
833. u833 More interestingly, Sipser [240] and Lautemann [166] proved that BPP is contained in ΣP 2
834. u834 2 (that is, the second level of PH). The Rabin-Miller and Solovay-Strassen algorithms imply that Primes ∈BPP.
835. u835 Today, most complexity theorists conjecture that what happened to Primes can happen to all of BPP: Conjecture 31 P = BPP.
836. u836 The reason for this conjecture is that it follows from the existence of good enough pseudorandom generators, which we could use to replace the random string r in any BPP algorithm M by deterministic strings that “look random, as far as M can tell.” Furthermore, work in the 1990s showed that, if we grant certain plausible lower bounds on circuit size, then these pseudorandom generators exist.
837. u837 Perhaps the most striking result along these lines is that of Impagliazzo and Wigderson [136]: Theorem 32 (Impagliazzo-Wigderson [136]) Suppose there exists a language decidable in 2n time, which requires nonuniform circuits of size 2Ω(n).
838. u838 Of course, if P = BPP, then the question of whether randomized algorithms can efficiently solve NP-complete problems is just the original P ?= NP question in a different guise. Ironically, however, the “obvious” approach to proving P = BPP is to prove a strong circuit lower bound—and if we knew how to do that, perhaps we could prove P̸ = NP as well!
839. u839 Even if we don’t assume P = BPP, it’s easy to show that deterministic nonuniform algorithms (see Section 5.2) can simulate randomized algorithms:
840. u840 38At least, not for arbitrary polynomials computed by small formulas or circuits. A great deal of progress has been made derandomizing PIT for restricted classes of polynomials.
841. u841 In fact, the deterministic primality test of Agrawal, Kayal, and Saxena [15] was based on a derandomization of one extremely special case of PIT.
842. u842 Proposition 33 (Adleman [12]) BPP ⊂P/poly.
843. u843 Proof. Let the language L be decided by a BPP algorithm that uses p (n) random bits.
844. u844 Then by using q (n) = O (n · p (n)) random bits, running the algorithm O (n) times with independent random bits each time, and outputting the majority answer, we can push the probability of error on any given input x ∈{0, 1}n from 1/3 down to (say) 2−2n.
845. u845 Thus, ( the probability that there on exists any an given x ∈{0, input 1}n x on ∈{0, which 1}n the from algorithm 1/3 down errs to (say) is at most 2−2n. 2n (
846. u846 This means, in particular, that there must be a fixed choice for the random string r ∈{0, 1}q(n) that causes the algorithm to succeed on all x ∈{0, 1}n. So to decide L in P/poly, we simply “hardwire” that r as the advice.
847. u847 By combining Theorem 26 with Proposition 33, we immediately obtain that if NP ⊆BPP, then the polynomial hierarchy collapses to the second level.
848. u848 So the bottom line is that the NP ⊆BPP question is likely identical to the P ?= NP question, but is extremely tightly related even if not.
849. u849 5.5
850. u850 Quantum Algorithms The class BPP might not exhaust what the physical world lets us efficiently compute, with quantum computing an obvious contender for going further.
851. u851 In 1993, Bernstein and Vazirani [47] defined the complexity class BQP, or Bounded-Error Quantum Polynomial-Time, as a quantum-mechanical generalization of BPP.
852. u852 (Details of quantum computing and BQP are beyond the scope of this survey, but see [203, 6].)
853. u853 Bernstein and Vazirani, along with Adleman, DeMarrais, and Huang [13], also showed some basic containments:
854. u854 P ⊆BPP ⊆BQP ⊆PP ⊆P#P ⊆PSPACE.
855. u855 In 1994, Shor [237] famously showed that the factoring and discrete logarithm problems are in BQP—and hence, that a scalable quantum computer, if built, could break almost all currently-used public-key cryptography.
856. u856 A more theoretical implication of Shor’s results is that, if factoring or discrete logarithm is hard classically, then BPP̸ = BQP.
857. u857 To design his quantum algorithms, Shor had to exploit extremely special properties of factoring and discrete logarithm, which aren’t known or believed to hold for NP-complete problems.
858. u858 ?= NP question is the question of whether NP ⊆BQP: that is, can quantum computers solve NP-complete problems in polynomial time? 39
859. u859 Most quantum computing researchers conjecture that the answer is no: Conjecture 34 NP̸ ⊂BQP.
860. u860 Naturally, there’s little hope of proving Conjecture 34 at present, since any proof would imply P̸ = NP!
861. u861 We don’t even know today how to prove conditional statements (analogous to what we have for BPP and P/poly): for example, that if NP ⊆BQP then PH collapses. On the other hand, it is known that, if a fast quantum algorithm for NP-complete problems exists, then in some sense it will have to be extremely different from Shor’s or any other known quantum algorithm.
862. u862 For example, Bennett et al. [45] showed that, if we ignore the structure of NP-complete problems, and just consider the abstract task of searching an unordered list, then quantum computers can provide
863. u863 39One can also consider the QMA-complete problems, which are a quantum generalization of the NP-complete problems themselves (see [54]), but we won’t pursue that here.
864. u864 at most a square-root speedup over the classical running time.
865. u865 As Bennett et al. [45] noted, this implies that there exists an oracle A such that NPA̸ ⊂BQPA. Note that the square-root speedup is actually achievable, using Grover’s algorithm [117].
866. u866 For most NP-complete problems, however, the fastest known quantum algorithm will be obtained by simply layering Grover’s algorithm on top of the fastest known classical algorithm, yielding a quadratic speedup but no more.40
867. u867 So for example, as far as anyone knows today, even a quantum computer would need 2Ω(n) time to solve 3Sat.
868. u868 Of course, one can also wonder whether the physical world might provide computational resources even beyond quantum computing (based on black holes? closed timelike curves? modifications to quantum mechanics?), and if so, whether those resources might enable the polynomial-time solution of NP-complete problems. If so, we might still have P̸ = NP—keep in mind that the classes P and NP have fixed mathematical definitions, which don’t change with the laws of physics—but we’d also have NP ⊆C, where C is whichever class of problems is efficiently solvable in the physical world.41
869. u869 Such speculations are beyond the scope of this article, but see for example [3].
870. u870 6
871. u871 Progress One common view among mathematicians is that questions like P ?= NP, while undoubtedly important, are just too hard to make progress on in the present state of mathematics. It’s true that we seem to be nowhere close to a solution, but in this section, I’ll build a case that the extreme pessimistic view is unwarranted. I’ll explain what genuine knowledge I think we have, relevant to proving P̸ = NP, that we didn’t have thirty years ago or in many cases ten years ago.
872. u872 One could argue that, if P̸ = NP is a distant peak, then all the progress has remained in the foothills.
873. u873 On the other hand, scaling the foothills has already been nontrivial, so anyone aiming for the summit had better get acquainted with what’s been done.
874. u874 More concretely, I’ll tell a story of the interaction between lower bounds and barriers: on the one hand, actual successes in proving superpolynomial or exponential lower bounds in interesting models of computation; but on the other, explanations for why the techniques used to achieve those successes don’t extend to prove P̸ = NP.
875. u875 We’ll see how the barriers influence the next generation of lower bound techniques, which are sometimes specifically designed to evade the barriers, or evaluated on their potential to do so.
876. u876 40One can artificially design an NP-complete problem with a superpolynomial quantum speedup over the best known classical algorithm by, for example, taking the language
877. u877 0φ0 · · · 0 | φ is a satisfiable 3Sat instance of size n0.01}
878. u878 {1x | x is a binary encoding of a positive integer with an odd number of distinct prime factors} . Clearly L is NP-complete, and a quantum algorithm can decide L in O
879. u879 time for some c, whereas the best known classical algorithm will take ∼exp
880. u880 Conversely, there are also NP-complete problems for which a quadratic quantum speedup is unknown—say, because the best known classical algorithm is based on dynamic programming, and it’s unclear how to combine that with Grover’s algorithm. One possible example is the Traveling Salesperson Problem, which is solvable in O (2n poly (n)) time using the Held-Karp dynamic √ programming algorithm [128], whereas a na¨ıve application of Grover’s algorithm yields only the worse bound O(
881. u881 As this survey was being completed, work by Moylett et al. [186] showed how to achieve a quadratic quantum speedup over the best known classical algorithms, for Traveling Salesperson on graphs of bounded degree 3 or 4. But it remains unclear how broadly these techniques can be applied.
882. u882 41For example, if the world is governed by quantum mechanics as physicists currently understand it, then C = BQP.
883. u883 With a single exception—namely, the Mulmuley-Sohoni Geometric Complexity Theory program— I’ll restrict my narrative to ideas that have already had definite successes in proving new limits on computation.
884. u884 The drawback of this choice is that in many cases, the ideas that are concrete enough to have worked for something, are also concrete enough that we understand why they can’t work for P̸ = NP! My defense is that this section would be unmanageably long, if it had to cover every idea about how P̸ = NP might someday be proved.
885. u885 I should, however, at least mention some important approaches to lower bounds that will be missing from my subsequent narrative. The first is descriptive complexity theory; see for example the book of Immerman [132] for the standard introduction, or Fagin’s personal perspective on finite model theory [86] for an approachable high-level overview.
886. u886 Descriptive complexity characterizes many complexity classes in terms of their logical expressive power: for example, P corresponds to sentences expressible in first-order logic with linear order and a least fixed point; NP to sentences expressible in existential second-order logic; PSPACE to sentences expressible in second-order logic with transitive closure; and EXP to sentences expressible in second-order logic with a least fixed point.
887. u887 The hope is that characterizing complexity classes in this way, with no explicit mention of resource bounds, might make it easier to see which are equal and which different.
888. u888 There are a few pieces of evidence for this hope.
889. u889 Namely, descriptive complexity played an important role in the proof by Immerman [131] that nondeterministic space is closed under complement (though the independent proof by Szelepcs´enyi [254] of the same result didn’t use these ideas).
890. u890 Descriptive complexity also played a role in the proof by Ajtai [17] that Parity is not in AC0 (Theorem 50 in Section 6.2.3, to follow)—though once again, the independent proof by Furst, Saxe, and Sipser [102] didn’t use these ideas.
891. u891 Finally, Fagin [85] was able to show that a class called “monadic NP”—which (alas) isn’t easy to define without the framework of finite model theory—is not closed under complement.
892. u892 On its face, this result looks tantalizingly close to NP̸ = coNP (but of course is much easier!). Whether descriptive complexity can lead to new separations between “ordinary” complexity classes, separations that aren’t as easily obtainable without it, remains to be seen.
893. u893 The second approach is lower bounds via communication complexity. Given a Boolean function f : {0, 1}n →{0, 1}, consider the following communication game: Alice receives an n-bit input x = x1 · · · xn such that f (x) = 0, Bob receives an input y such that f (y) = 1, and their goal is to agree on an index i ∈{1, . . . , n} such that xi̸ = yi.
894. u894 Let Cf be the communication complexity of this game: that is, the minimum number of bits that Alice and Bob need to exchange to win the game, if they use an optimal protocol (and where the communication cost is maximized over all x, y pairs). Then in 1990, Karchmer and Wigderson [144] showed the following remarkable connection. Theorem 35 (Karchmer-Wigderson [144]) For any f, the minimum depth of any Boolean circuit for f is equal to Cf.
895. u895 Combined with Proposition 27 (depth-reduction for formulas), Theorem 35 implies that every Boolean function f requires formulas of size at least 2Cf : in other words, communication lower bounds imply formula-size lower bounds.
896. u896 Now, communication complexity is a well-established area of theoretical computer science with many strong lower bounds; see for example the book by Kushilevitz and Nisan [159].
897. u897 Thus, we might hope that lower-bounding the communication cost of the “Karchmer-Wigderson game,” say for an NP-complete problem like HamiltonCycle, could be a viable approach to proving NP̸ ⊂NC1, which in turn would be a huge step toward P̸ = NP.
898. u898 See Section 6.2.2 for Karchmer and Wigderson’s applications of a similar connection to monotone formula-size lower bounds.
899. u899 Also see Aaronson and Wigderson [10] for further connections between communication complexity and computational complexity, including even a “communication complexity lower bound” that if true would imply P̸ = NP. Of course, the question is whether these translations merely shift the difficulty of complexity class separations to a superficially different setting, or whether they set the stage for genuinely new insights.
900. u900 The third approach is lower bounds via derandomization.
901. u901 In Section 5.4.1, we discussed the discovery in the 1990s that, if sufficiently strong circuit lower bounds hold, then P = BPP: that is, every randomized algorithm can be made deterministic with only a polynomial slowdown.
902. u902 In the early 2000s, it was discovered that converse statements often hold as well: that is, derandomizations of randomized algorithms imply circuit lower bounds. Probably the best-known result along these lines is that of Kabanets and Impagliazzo [140]: Theorem 36 ([140]) Suppose the polynomial identity testing problem from Section 5.4 is in P. Then either NEXP̸ ⊂P/poly, or else the permanent function has no polynomial-size arithmetic circuits (see Section 6.5.1).
903. u903 As usual, the issue is that it’s not clear whether we should interpret this result as giving a plausible path toward proving circuit lower bounds (namely, by derandomizing PIT), or simply as explaining why derandomizing PIT will be hard (namely, because doing so will imply circuit lower bounds)!
904. u904 In any case, Sections 6.3.2 and 6.4.2 will give further examples where derandomization results would imply new circuit lower bounds.
905. u905 The fourth approach could be called lower bounds via “innocent-looking” combinatorics problems. Here’s an example: given an n × n matrix A, say over the finite ( field F2, call A rigid if not only does A have rank Ω(n), but any matrix obtained by changing O
906. u906 entries in each row of A also has rank Ω(n).
907. u907 It’s easy to show, via a counting argument, that almost all matrices A ∈Fn×n 2
908. u908 On the other hand, Valiant [260] made the following striking observation in 1977: if we manage to find any explicit example of a rigid matrix, then we also get an explicit example of a Boolean function that can’t be computed by any circuit of linear size and logarithmic depth.
909. u909 For another connection in the same spirit, given a 3-dimensional tensor A ∈Fn×n×n 2
910. u910 , let the rank of A be the smallest r such that A can be written as the sum of r rank-one tensors (that is, tensors of the form tijk = xiyjzk). ( Then it’s easy to show, via a counting argument, that almost all tensors A ∈Fn×n×n 2
911. u911 On the other hand, Strassen [251] observed in 1973 that, if we find any explicit example of a 3-dimensional tensor with rank r, then we also get an explicit example of a Boolean function with circuit complexity Ω(r).42
912. u912 Alas, proving that any explicit matrix is rigid, or that any explicit tensor has superlinear rank, have turned out to be staggeringly hard problems—as perhaps shouldn’t surprise us, given the implications for circuit lower bounds!
913. u913 The rest of the section is organized as follows:
914. u914 42Going even further, Raz [216] proved in 2010 that, if we manage to show that any explicit d-dimensional tensor A : [n]d →F has rank at least nd(1−o(1)), then we’ve also shown that the n×n permanent function has no polynomialsize arithmetic formulas. It’s easy to construct explicit d-dimensional tensors with rank n⌊d/2⌋, but the current record is an explicit d-dimensional tensor with rank at least 2n⌊d/2⌋+ n −O (d log n) [19].
915. u915 Note that, if we could show that the permanent had no nO(log n)-size arithmetic formulas, that would imply Valiant’s famous Conjecture 72: that the permanent has no polynomial-size arithmetic circuits.
916. u916 However, Raz’s technique seems incapable of proving formula-size lower bounds better than nΩ(log log n).
917. u917 Section 6.1 covers logical techniques, which typically fall prey to the relativization barrier.
918. u918 Section 6.2 covers combinatorial techniques, which typically fall prey to the natural proofs
919. u919 Section 6.3 covers “hybrid” techniques (logic plus arithmetization), many of which fall prey
920. u920 Sections 6.4 covers “ironic complexity theory” (as exemplified by the recent work of Ryan
921. u921 Williams), or the use of nontrivial algorithms to prove circuit lower bounds.
922. u922 Section 6.5 covers arithmetic circuit lower bounds, which probably fall prey to arithmetic
923. u923 variants of the natural proofs barrier (though this remains disputed).
924. u924 Section 6.6 covers Mulmuley and Sohoni’s Geometric Complexity Theory (GCT), an auda-
925. u925 ?= NP and related problems by reducing them to questions in
926. u926 algebraic geometry and representation theory (and which is also an example of “ironic com-
927. u927 Note that, for the approaches covered in Sections 6.4 and 6.6, no formal barriers are yet known.
928. u928 6.1
929. u929 Logical Techniques In the 1960s, Hartmanis and Stearns [122] realized that, by simply “scaling down” Turing’s diagonalization proof of the undecidability of the halting problem, we can at least prove some separations between complexity classes.
930. u930 In particular, we can generally show that more of the same resource (time, memory, etc.) lets us decide more languages than less of that resource.
931. u931 Here’s a special case of their so-called Time Hierarchy Theorem.
932. u932 Theorem 37 (Hartmanis-Stearns [122]) P is strictly contained in EXP.
933. u933 Proof. Let
934. u934 L = {(⟨M⟩, x, 0n) : M (x) halts in at most 2n steps} .
935. u935 Clearly L ∈EXP.
936. u936 On the other hand, suppose by contradiction that L ∈P.
937. u937 Then there’s some polynomial-time Turing machine A such that A (z) accepts if and only if z ∈L.
938. u938 Let A run in p (n + |⟨M⟩| + |x|) time.
939. u939 Then using A, we can easily produce another machine B that does the following:
940. u940 Runs forever if M (⟨M⟩, 0n) halts in at most 2n steps; otherwise halts.
941. u941 Note that, if B halts at all, then it halts after only p (2n + 2 |⟨M⟩|) = nO(1) steps.
942. u942 Now consider what happens when B is run on input (⟨B⟩, 0n).
943. u943 If B (⟨B⟩, 0n) runs forever, then B (⟨B⟩, 0n) halts. Conversely, if B (⟨B⟩, 0n) halts, then for all sufficiently large n, it halts in fewer than 2n steps, but that means that B (⟨B⟩, 0n) runs forever.
944. u944 So we conclude that B, and hence A, can’t have existed.
945. u945 More broadly, the same ( argument shows that there ( are languages decidable in O
946. u946 time but not in O (n) time, in O
947. u947 time, and so on for almost every natural pair of runtime bounds. (Technically, we have TIME (f (n))̸ = TIME (g (n)) for every f, g that are time-constructible—that is, there exist Turing machines that run for f (n) and g (n) steps given n as input—and that are separated by more than a log n multiplicative factor.)
948. u948 Likewise, the Space Hierarchy Theorem shows that there are languages decidable in O (f (n)) space but not in O (g (n)) space, for all natural f (n) ≫g (n).
949. u949 Cook [77] also proved a hierarchy theorem for the nondeterministic time classes, which will play an important role in Section 6.4:
950. u950 Theorem 38 (Nondeterministic Time Hierarchy Theorem [77]) For all time-constructible
951. u951 f, g such that f (n + 1) = o (g (n)), we have that NTIME (f (n)) is strictly contained in NTIME (g (n)).
952. u952 One amusing consequence of the hierarchy theorems is that we can prove, for example, that P̸ = SPACE (n), even though we can’t prove either that P̸ ⊂SPACE (n) or that SPACE (n)̸ ⊂P! For suppose by contradiction that P ( = SPACE (n). Then by a padding ( argument (cf. Proposition 17), P would also contain ( SPACE
953. u953 But then we’d have SPACE (n) = SPACE
954. u954 , violating the Space Hierarchy Theorem.
955. u955 Most computer scientists conjecture both that P̸ ⊂SPACE (n) and that SPACE (n)̸ ⊂P, but proving either statement by itself is a much harder problem.
956. u956 In summary, there really is a rich, infinite hierarchy of harder and harder computable problems. Complexity classes don’t collapse in the most extreme ways imaginable, with (say) everything solvable in linear time.
957. u957 6.1.1
958. u958 Circuit Lower Bounds Based on Counting A related idea—not exactly “diagonalization,” but counting arguments made explicit—can also be used to show that certain problems can’t be solved by polynomial-size circuits.
959. u959 This story starts with Claude Shannon [235], who made the following fundamental observation in 1949. Proposition 39 (Shannon [235]) There exists a Boolean function f : {0, 1}n →{0, 1}, on n variables, such that any circuit to compute f requires at least Ω(2n/n) logic gates.
960. u960 Indeed, almost all Boolean functions on n variables (that is, a 1 −o (1) fraction of them) have this property.43 Proof. There are 22n different Boolean functions f on n variables, but only
961. u961 different Boolean circuits with n inputs and at ( most T NAND gates.
962. u962 Since each circuit only represents one function, and since (n + T)2T = o
963. u963 when T = o (2n/n), it follows by a counting argument (i.e., the pigeonhole principle) that some f must require a circuit with T = Ω(2n/n) NAND gates—and indeed, that almost all of the 22n possible f’s must have this property.
964. u964 The number of AND, OR, and NOT gates required is related to the number of NAND gates by a constant factor, so is also Ω(2n/n).
965. u965 43With some effort, Shannon’s lower bound can be shown to be tight: that is, every n-variable Boolean function can be represented by a circuit of size O (2n/n). (The obvious upper bound is O (n2n).)
966. u966 Famously, Proposition 39 shows that there exist Boolean functions that require exponentially large circuits—in fact, that almost all of them do—yet it fails to produce a single example of such a function! It tells us nothing whatsoever about 3Sat or Clique or any other particular function that might interest us.
967. u967 In that respect, it’s similar to Shannon’s celebrated proof that almost all codes are good error-correcting codes, which also fails to produce a single example of such a code. Just like, in the decades after Shannon, the central research agenda of coding theory was to “make Shannon’s argument explicit” by finding specific good error-correcting codes, so too the agenda of circuit complexity has been to “make Proposition 39 explicit” by finding specific functions that provably require large circuits.
968. u968 In some cases, the mere fact that we know, from Proposition 39, that hard functions exist lets us “bootstrap” to show that particular complexity classes must contain hard functions. Here’s an example of this.
969. u969 Theorem 40 EXPSPACE̸ ⊂P/poly.
970. u970 Then by Proposition 39, there exist functions f : {0, 1}n → Proof. Let n be sufficiently large.
971. u971 Thus, if we list all the 22n {0, 1} with circuit complexity at least c2n/n, for some constant c > 0.
972. u972 functions in lexicographic order by their truth tables, there must be a first function in the list, call it fn, with circuit complexity at least c2n/n. We now define
973. u973 Then by construction, L /∈P/poly.
974. u974 On the other hand, enumerating all n-variable Boolean functions, calculating the circuit complexity of each, and finding the first one with circuit complexity at least c2n/n can all be done in exponential space. Hence L ∈EXPSPACE.
975. u975 There’s also a “scaled-down version” of Theorem 40, proved in the same way: Theorem 41 For every fixed k, there is a language in PSPACE that does not have circuits of size nk.44
976. u976 By being a bit more clever, Kannan [143] lowered the complexity class in Theorem 40 from EXPSPACE to NEXPNP.
977. u977 Theorem 42 (Kannan [143]) NEXPNP̸ ⊂P/poly.
978. u978 Proof. First, we claim that EXPNPNP̸ ⊂P/poly. The reason is simply a more careful version of the proof of Theorem 40: in EXPNPNP, we can do an explicit binary search for the lexicographically first Boolean function fn : {0, 1}n →{0, 1} such that every circuit of size at most (say) c2n/n disagrees with fn on some input x. (Such an fn must exist by a counting argument.)
979. u979 Next, suppose by contradiction that NEXPNP ⊂P/poly. Then certainly NP ⊂P/poly. By the
980. u980 2 , so in particular PNPNP = NPNP. Karp-Lipton Theorem (Theorem 26), this implies that PH = ΣP 2 By upward translation (as in Proposition 17), this in turn means 2 that EXPNPNP = NEXPNP.
981. u981 But we already know that EXPNPNP doesn’t have polynomial-size circuits, and therefore neither does NEXPNP.
982. u982 44Crucially, this will be a different language for each k; otherwise we’d get PSPACE̸ ⊂P/poly, which is far beyond our current ability to prove.
983. u983 Amusingly, if we work out the best possible lower bound that we can get from Theorem 42 on the circuit complexity of a language in NEXPNP, it turns out to be half-exponential: that is, a function f such that f (f (n)) grows exponentially.
984. u984 Such functions exist, but have no closed-form expressions.
985. u985 Directly analogous to Theorem 41, a “scaled-down” version of the proof of Theorem 42 shows
986. u986 2 = NPNP that doesn’t have circuits of size nk. that, for every fixed k, there’s a language in ΣP 2
987. u987 In Section 6.3, we’ll discuss slight improvements to these results that can be achieved with algebraic methods.
988. u988 Nevertheless, it (sadly) remains open even to show that NEXP̸ ⊂P/poly, or that there’s a language in NP that doesn’t have linear-sized circuits.
989. u989 6.1.2
990. u990 The Relativization Barrier The magic of diagonalization, self-reference, and counting arguments is how abstract and general they are: they never require us to “get our hands dirty” by understanding the inner workings of algorithms or circuits.
991. u991 But as was recognized early in the history of complexity theory, the price of generality is that the logical techniques are extremely limited in scope.
992. u992 Often the best way to understand the limits of a proposed approach for proving a statement S, is to examine what else besides S the approach would prove if it worked—i.e., which stronger statements S′ the approach “fails to differentiate” from S.
993. u993 If any of the stronger statements are false, then the approach can’t prove S either.
994. u994 That is exactly what Baker, Gill, and Solovay [38] did for diagonalization in 1975, when they articulated the relativization barrier.
995. u995 Their central insight was that almost all the techniques we have for proving statements in complexity theory—such as C ⊆D or C̸ ⊂D, where C and D are two complexity classes—are so general that, if they work at all, then they actually prove CA ⊆DA or CA̸ ⊂DA for all possible oracles A. In other words: if all the machines that appear in the proof are enhanced in the same way, by being given access to the same oracle, the proof is completely oblivious to that change, and goes through just as before.
996. u996 A proof with this property is said to “relativize,” or to hold “in all possible relativized worlds” or “relative to any oracle.”
997. u997 Intuitively, because the proofs only do things like using one Turing machine M1 to simulate a second Turing machine M2 step-by-step, without examining either machine’s internal structure. In that case, if M2 is given access to an oracle A, then M1 can still simulate M2 just fine, provided that M1 is also given access to A, in order to simulate M2’s oracle calls.
998. u998 To illustrate, you might want to check that the proofs of Theorems 37, 40, and 42 can be straightforwardly modified to show that, more generally:
999. u999 EXPSPACEA̸ ⊂PA/poly for all oracles A.
1000. u1000 NEXPNPA̸ ⊂PA/poly for all oracles A.
1001. u1001 Alas, Baker, Gill, and Solovay then observed that no relativizing technique can possibly resolve
1002. u1002 For, unlike (say) P ?= EXP or the unsolvability of the halting problem, the P P ?= NP admits “contradictory relativizations”: there are some oracle worlds where P = NP, and others where P̸ = NP. For that reason, any proof of P̸ = NP will need to “notice,” at some point,
1003. u1003 that there are no oracles in “our” world: it will have to use techniques that fail relative to certain oracles.
1004. u1004 Theorem 43 (Baker-Gill-Solovay [38]) There exists an oracle A such that PA = NPA, and another oracle B such that PB̸ = NPB.
1005. u1005 Proof Sketch. To make PA = NPA, we can just let A be any PSPACE-complete language. Then it’s not hard to see that PA = NPA = PSPACE.
1006. u1006 To make PB̸ = NPB, we can (for example) let B be a random oracle, as observed by Bennett and Gill [46]. We can then, for example, define
1007. u1007 L = {0n : the first 2n bits returned by B contain a run of n consecutive 1’s} .
1008. u1008 By contrast, one can easily show that L /∈PB with probability 1 over B: in Clearly L ∈NPB.
1009. u1009 this case, there really is nothing for a deterministic Turing machine to do but brute-force search, requiring exponentially many queries to the B oracle.
1010. u1010 We also have the following somewhat harder result.
1011. u1011 Theorem 44 (Wilson [282]) There exists an oracle A such that NEXPA ⊂PA/poly, and such that every language in NPA has linear-sized circuits with A-oracle gates (that is, gates that query A).
1012. u1012 In other words, any proof even of NEXP̸ ⊂P/poly—that is, of a circuit lower bound just “slightly” beyond those that have already been proven—will require non-relativizing techniques. One can likewise show that non-relativizing techniques will be needed to make real progress on many of the other open problems of complexity theory (such as proving P = BPP).
1013. u1013 If relativization seems too banal, the way to appreciate it is to try to invent techniques, for proving inclusions or separations among complexity classes, that fail to relativize.
1014. u1014 It’s harder than it sounds! A partial explanation for this was given by Arora, Impagliazzo, and Vazirani [30], who reinterpreted the relativization barrier in logical terms. From their perspective, a relativizing proof is simply any proof that “knows” about complexity classes, only through axioms that assert the classes’ closure properties, as well as languages that the classes do contain.
1015. u1015 (For example, P contains the empty language; if L1 and L2 are both in P, then so are Boolean combinations like L1 and L1 ∩L2.)
1016. u1016 These axioms can be shown to imply statements such as P̸ = EXP.
1017. u1017 But other statements, like P̸ = NP, can be shown to be independent of the axioms, by constructing models of the axioms where those statements are false.
1018. u1018 One constructs those models by using oracles to “force in” additional languages—such as PSPACE-complete languages, if one wants a world where P = NP—which the axioms might not require to be contained in complexity classes like P and NP, but which they don’t prohibit from being contained, either.
1019. u1019 The conclusion is that any proof of P̸ = NP will need to appeal to deeper properties of the classes P and NP, properties that don’t follow from these closure axioms.
1020. u1020 6.2
1021. u1021 Combinatorial Lower Bounds Partly because of the relativization barrier, in the 1980s attention shifted to combinatorial approaches: that is, approaches where one tries to prove superpolynomial lower bounds on the number of operations of some kind needed to do something, by actually “rolling up one’s sleeves” and
1022. u1022 delving into the messy details of what the operations do (rather than making abstract diagonalization arguments).
1023. u1023 These combinatorial approaches enjoyed some spectacular successes, some of which seemed at the time like they were within striking distance of proving P̸ = NP.
1024. u1024 Let’s see some examples.
1025. u1025 6.2.1
1026. u1026 Proof Complexity Suppose we’re given a 3Sat formula φ, and we want to prove that φ has no satisfying assignments. One natural approach to this is called resolution: we repeatedly pick two clauses of φ, and then “resolve” the clauses (or “smash them together”) to derive a new clause that logically follows from the first two.
1027. u1027 This is most useful when one of the clauses contains a non-negated literal x, and the other contains the corresponding negated literal x. For example, from the clauses (x ∨y) and (x ∨z), it’s easy to see that we can derive (y ∨z).
1028. u1028 The new derived clause can then be added to the list of clauses, and used as an input to future resolution steps.
1029. u1029 Now, if we ever derive the empty clause ( )—say, by smashing together (x) and (x)—then we can conclude that our original 3Sat formula φ must have been unsatisfiable.
1030. u1030 For in that case, φ entails a clause that’s not satisfied by any setting of variables.
1031. u1031 Another way to say this is that resolution is a sound proof system.
1032. u1032 By doing induction on the number of variables in φ, it’s not hard to show that resolution is also complete: Proposition 45 Resolution is a complete proof system for the unsatisfiability of kSat.
1033. u1033 In other words, given any unsatisfiable kSat formula φ, there exists some sequence of resolution steps that produces the empty clause.
1034. u1034 So the key question about resolution is just how many resolution steps are needed to derive the empty clause, starting from an unsatisfiable formula φ.
1035. u1035 If that number could be upper-bounded by a polynomial in the size of φ, it would follow that NP = coNP.
1036. u1036 If, moreover, an appropriate sequence of resolutions could actually be found in polynomial time, it would follow that P = NP.
1037. u1037 On the other hand, when we prove completeness by induction on the number of variables n, the only upper bound we get on the number of resolution steps is 2n. And indeed, in 1985, Haken proved the following celebrated result.
1038. u1038 Theorem 46 (Haken [121]) There exist kSat formulas, involving nO(1) variables and clauses, for which any resolution proof of unsatisfiability requires at least 2Ω(n) resolution steps. An example is a kSat formula that explicitly encodes the “nth Pigeonhole Principle”: that is, the statement that there’s no way to assign n + 1 pigeons to n holes, without assigning two or more pigeons to the same hole.
1039. u1039 Haken’s proof formalized the intuition that any resolution proof of the Pigeonhole Principle will ultimately be stuck “reasoning locally”: “let’s see, if I put this pigeon there, and that one there ... darn, it still doesn’t work!” Such a proof has no ability to engage in higher-level reasoning about the total number of pigeons.45
1040. u1040 45Haken’s proof has since been substantially simplified. Readers interested in a “modern” version should consult Ben-Sasson and Wigderson [44], who split the argument into two easy steps: first, “short proofs are narrow”—that is, any short resolution proof can be converted into another such proof that only contains clauses with few literals—and second, any resolution refutation of a kSat instance encoding (for example) the pigeonhole principle must be “wide,” because of the instance’s graph-expansion properties.
1041. u1041 Since Haken’s breakthrough, there have been many other exponential lower bounds on the sizes of unsatisfiability proofs, typically for proof systems that generalize resolution in some way (see Beame and Pitassi [40] for a good survey).
1042. u1042 These, in turn, often let us prove exponential lower bounds on the running times of certain kinds of algorithms.
1043. u1043 For example, there’s a widely-used class of kSat algorithms called DPLL (Davis-Putnam-Logemann-Loveland) algorithms [82], which are based on pruning the search tree of possible satisfying assignments. DPLL algorithms have the property that, if one looks at the search tree of their execution on an unsatisfiable kSat formula φ, one can read offa resolution proof that φ is unsatisfiable. From that fact, together with Theorem 46, it follows that there exist kSat formulas (for example, the Pigeonhole Principle formulas) for which any DPLL algorithm requires exponential time.
1044. u1044 In principle, if one could prove superpolynomial lower bounds for arbitrary proof systems (constrained only by the proofs being checkable in polynomial time), one would get P̸ = NP, and even NP̸ = coNP! However, perhaps this motivates turning our attention to lower bounds on circuit size, which tend to be somewhat easier than the analogous proof complexity lower bounds, and which—if generalized to arbitrary Boolean circuits—would “merely” imply P̸ = NP and NP̸ ⊂P/poly, rather than NP̸ = coNP.
1045. u1045 6.2.2
1046. u1046 Monotone Circuit Lower Bounds Recall, from Section 5.2, that if we could merely prove that any family of Boolean circuits to solve some NP problem required a superpolynomial number of AND, OR, and NOT gates, then that would imply P̸ = NP, and even the stronger result NP̸ ⊂P/poly (that is, NP-complete problems are not efficiently solvable by nonuniform algorithms).
1047. u1047 Now, some NP-complete languages L have the interesting property of being monotone: that is, changing an input bit from 0 to 1 can change the answer from x /∈L to x ∈L, but never from x ∈L
1048. u1048 An example is the Clique language: say, the set of all encodings of n-vertex graphs to x /∈L.
1049. u1049 G, to x as /∈L. adjacency An example matrices is of the 0s and 1s, such language: that G say, contains the set a clique of all encodings on at least of √n n-vertex vertices.
1050. u1050 It’s not hard to see that we can decide any such language using a monotone circuit: that is, a Boolean
1051. u1051 For the Clique language, for example, a circuit of AND and OR gates only, no ( NOT ) gates.
1052. u1052 circuit could simply consist of an OR of
1053. u1053 ANDs, one for each possible clique. It thus becomes interesting to ask what are the smallest monotone circuits for monotone NP-complete languages.
1054. u1054 In 1985, Alexander Razborov, then a graduate student, astonished the complexity theory world with the following result.
1055. u1055 Theorem 47 (Razborov [219]) Any monotone circuit for Clique requires at least nΩ(log n) gates.
1056. u1056 Subsequently, Alon and Boppana [27] improved this, ( to ( show that any monotone circuit to detect a clique of size ∼(n/ log n)2/3 must have size exp
1057. u1057 I won’t go into the proof of Theorem 47 here, but it uses beautiful combinatorial techniques, including (in modern versions) the Erd˝os-Rado sunflower lemma.
1058. u1058 The significance of Theorem 47 is this: if we could now merely prove that any circuit for a monotone language can be made into a monotone circuit without much increasing its size, then we’d immediately get P̸ = NP and even NP̸ ⊂P/poly.
1059. u1059 Indeed, this was considered a potentiallyviable approach to proving P̸ = NP for some months. Alas, the approach turned out to be a dead end, because of a result first shown by Razborov himself (and then improved by Tardos):
1060. u1060 Theorem 48 (Razborov [220], Tardos [256]) There are monotone languages in P that require exponentially-large monotone circuits.
1061. u1061 An example is the Matching language, consisting of all adjacency-matrix encodings of n-vertex graphs that admit a matching ( ( on at least ∼(n/ log n)2/3 vertices.
1062. u1062 This language requires monotone circuits of size exp
1063. u1063 Thus, while Theorem 47 stands as a striking example of the power of combinatorics to prove circuit lower bounds, ultimately it tells us not about the hardness of NP-complete problems, but only about the weakness of monotone circuits.
1064. u1064 Theorem 48 implies that, even if we’re trying to compute a monotone Boolean function (such as the Matching function), allowing ourselves the non-monotone NOT gate can yield an exponential reduction in circuit size.
1065. u1065 But Razborov’s techniques break down completely as soon as a few NOT gates are available.46
1066. u1066 In the stCon (s, t-connectivity)
1067. u1067 I should also mention lower bounds on monotone depth.
1068. u1068 problem, we’re given as input the adjacency matrix of an undirected graph, and asked whether or not there’s a path between two designated vertices s and t.
1069. u1069 By using their connection between circuit depth and communication complexity (see Section 6), ( Karchmer ) and Wigderson [144] were able to prove that any monotone circuit for stCon requires Ω
1070. u1070 depth—and as a consequence, that any monotone formula for stCon requires nΩ(log n) size.
1071. u1071 Since stCon is known to have monotone circuits of polynomial size, this implies in particular that monotone formula size and monotone circuit size are not polynomially related.
1072. u1072 6.2.3
1073. u1073 Small-Depth Circuits and the Random Restriction Method
1074. u1074 Besides restricting the allowed gates (say, to AND and OR only), there’s a second natural way to “hobble” a circuit, and thereby potentially make it easier to prove lower bounds on circuit size. Namely, we can restrict the circuit’s depth, the number of layers of gates between input and output.
1075. u1075 If the allowed gates all have a fanin of 1 or 2 (that is, they all take only 1 or 2 input bits), then clearly any circuit that depends nontrivially on all n of the input bits must have depth at least log2 n.
1076. u1076 On the other hand, if we allow gates of unbounded fanin—for example, ANDs or XORs or MAJORITYs on unlimited numbers of inputs—then it makes sense to ask what can be computed even by circuits of constant depth.
1077. u1077 Constant-depth circuits are very closely related to neural networks, which also consist of a small number of layers of “logic gates” (i.e., the neurons), with each neuron allowed to have very large “fanin”—i.e., to accept input from many or all of the neurons in the previous layer.
1078. u1078 If we don’t also restrict the number of gates or neurons, then it turns out that every function can be computed in small depth: Proposition 49 Every Boolean function f : {0, 1}n →{0, 1} can be computed by an unboundedfanin, depth-3 circuit of size O (n2n): namely, by an OR of ANDs of input bits and their negations.
1079. u1079 We simply need to check whether the input, x ∈{0, 1}n, is one of the z’s such that Proof.
1080. u1080 46Note that, if we encode the input string using the so-called dual-rail representation—in which every 0 is represented by the 2-bit string 01, and every 1 by 10—then the monotone circuit complexities of Clique, Matching, and so on do become essentially equivalent to their non-monotone circuit complexities, since we can push all the NOT gates to the bottom layer of the circuit using de Morgan’s laws, and then eliminate the NOT gates using the dual-rail encoding. Unfortunately, Razborov’s lower bound techniques also break down under dual-rail encoding.
1081. u1081 f (z) = 1:
1082. u1082 Similarly, in typical neural network models, every Boolean function can be computed by a network with ∼2n neurons arranged into just 2 layers.
1083. u1083 So the interesting question is what happens if we restrict both the depth and the number of
1084. u1084 More formally, let AC0 be the class of languages L ⊆{0, 1}∗for which there gates or neurons.
1085. u1085 exists a family of circuits {Cn}n≥1, one for each input size n, such that:
1086. u1086 (1) Cn (x) outputs 1 if x ∈L and 0 if x /∈L, for all n and x ∈{0, 1}n.
1087. u1087 (2) Each Cn consists of unbounded-fanin AND and OR gates, as well as NOT gates.
1088. u1088 (3) There is a polynomial p such that each Cn has at most p (n) gates.
1089. u1089 (4) There is a constant d such that each Cn has depth at most d.
1090. u1090 Clearly AC0 is a subclass of P/poly; indeed we recover P/poly by omitting condition (4). Now, one of the major triumphs of complexity theory in the 1980s was to understand AC0, as we still only dream of understanding P/poly.
1091. u1091 It’s not just that we know NP̸ ⊂AC0; rather, it’s that we know in some detail which problems are and aren’t in AC0 (even problems within P), and in some cases, the exact tradeoffbetween the number of gates and the depth d. As the most famous example, let Parity be the language consisting of all strings with an odd number of ‘1’ bits. Then: Theorem 50 (Ajtai [17], Furst-Saxe-Sipser [102]) Parity is not in AC0.
1092. u1092 While the original lower bounds on the size of AC0 circuits for Parity were only slightly superpolynomial, Theorem 50 was subsequently improved by Yao [284] and then by H˚astad [123], the latter of whom gave an essentially optimal result: namely, any AC0 circuit for Parity of depth d requires at least 2Ω(n1/(d−1)) gates.
1093. u1093 The first proofs of Theorem 50 used what’s called the method of random restrictions.
1094. u1094 In this method, we assume by contradiction that we have a size-s, depth-d, unbounded-fanin circuit C for our Boolean function—say, the Parity function. We then randomly fix most of the input bits to 0 or 1, while leaving a few input bits unfixed.
1095. u1095 What we hope to find is that the random restriction “kills off” an entire layer of gates—because any AND gate that takes even one constant 0 bit as input can be replaced by the constant 0 function, and likewise, any OR gate that takes even one 1 bit as input can be replaced by the constant 1 function. Thus, any AND or OR gate with a large fanin is extremely likely to be killed off; gates with small fanin might not be killed off, but can be left around to be dealt with later.
1096. u1096 We then repeat this procedure, randomly restricting most of the remaining unfixed bits, in order to kill offthe next higher layer of AND and OR gates, and so on through all d layers.
1097. u1097 By the time we’re done, we’ve reduced C to a shadow of its former self: specifically, to a circuit that depends on only a constant number of input bits.
1098. u1098 Meanwhile, even though only a tiny fraction of the input bits (say, n1/d of them) remain unfixed, we still have a nontrivial Boolean function on those bits: indeed, it’s easy to see that any restriction of the Parity
1099. u1099 function to a smaller set of bits will either be Parity itself, or else NOT(Parity). But a circuit of constant size clearly can’t compute a Boolean function that depends on ∼n1/d input bits. This yields our desired contradiction.
1100. u1100 At a high level, there were three ingredients needed for the random restriction method to work. First, the circuit needed to be built out of AND and OR gates, which are likely to get killed offby random restrictions.
1101. u1101 The method wouldn’t have worked if the circuit contained unbounded-fanin MAJORITY gates (as a neural network does), or even unbounded-fanin XOR gates.
1102. u1102 Second, it was crucial that the circuit depth d was small, since we needed to shrink the number of unfixed input variables by a large factor d times, and then still have unfixed variables left ( over. It ) turns out that random restriction arguments can yield some lower bound whenever d = o
1103. u1103 , but not beyond that.
1104. u1104 Third, we needed to consider a function, such as Parity, that remains nontrivial even after the overwhelming majority of input bits are randomly fixed to 0 or 1.
1105. u1105 The method wouldn’t have worked, for example, for the n-bit AND function (which is unsurprising, since the AND function does have a depth-1 circuit, consisting of a single AND gate!).
1106. u1106 The original proofs for Parity /∈AC0 have been generalized and improved on in many ways. For example, Linial, Mansour, and Nisan [171] examined the weakness of AC0 circuits from a different angle: “turning lemons into lemonade,” they gave a quasipolynomial-time algorithm to learn arbitrary AC0 circuits with respect to the uniform distribution over inputs.47
1107. u1107 Also, proving a conjecture put forward by Linial and Nisan [172] (and independently Babai), Braverman [58] showed that AC0 circuits can’t distinguish the outputs of a wide range of pseudorandom generators from truly random strings.
1108. u1108 Meanwhile, H˚astad [123] showed that for every d, there are functions computable by AC0 circuits of depth d that require exponentially many gates for AC0 circuits of depth d−1. This implies that there exists an oracle relative to which PH is infinite (that is, all its levels are distinct). Improving that result, Rossman, Servedio, and Tan [226] very recently showed that the same functions H˚astad had considered require exponentially many gates even to approximate using AC0 circuits of depth d−1. This implies that PH is infinite relative to a random oracle with probability 1, which resolved a thirty-year-old open problem.
1109. u1109 The random restriction method has also had other applications in complexity theory, besides to AC0.
1110. u1110 Most notably, it’s been used to prove polynomial lower bounds on formula size.
1111. u1111 The story of formula-size lower bounds starts in 1961 with Subbotovskaya [252], who ( used random restrictions to show that the n-bit Parity ( function requires formulas of size Ω
1112. u1112 Later Khrapchenko [152] improved this to Ω
1113. u1113 Next, in 1987, Andreev [28] constructed a different Boolean function in P that could be shown, again using random restrictions, to require formulas of size n2.5−o(1). This was subsequently improved to n2.55−o(1) by Impagliazzo and Nisan ( [135], to n2.63−o(1) ) by Paterson and Zwick [206], and finally to n3−o(1) by H˚astad [124] and to Ω
1114. u1114 Unfortunately, the random restriction method seems fundamentally incapable of going beyond Ω
1115. u1115 On the other hand, for Boolean circuits rather than formulas, we still have no lower bound better than linear for any function in P (or for that
1116. u1116 47By a “learning algorithm,” here we mean an algorithm that takes as input uniformly-random samples x1, . . . , xk ∈ {0, 1}n, as well as f (x1) , . . . , f (xk), where f is some unknown function in AC0, and that with high probability over the choice of x1, . . . , xk, outputs a hypothesis h such that Prx∈{0,1}n [h (x) = f (x)] is close to 1.
1117. u1117 48Assume for simplicity that n is a power of 2. Then x1 ⊕· · ·⊕xn can be written as y⊕z, where y := x1 ⊕· · ·⊕xn/2 and z := xn/2+1 ⊕· · · ⊕xn.
1118. u1118 This in turn can be written as (y ∧z) ∨(y ∧z).
1119. u1119 Expanding recursively now yields a size-n2 formula for Parity, made of AND, OR, and NOT gates.
1120. u1120 matter, in NP)!
1121. u1121 6.2.4
1122. u1122 Small-Depth Circuits and the Polynomial Method For our purposes, the most important extension of Theorem 50 was achieved by Smolensky [243] and Razborov [221] in 1987. Let AC0 [m] be the class of languages decidable by a family of constantdepth, polynomial-size, unbounded-fanin circuits with AND, OR, NOT, and MOD-m gates (which output 1 if their number of ‘1’ input bits is divisible by m, and 0 otherwise).
1123. u1123 Adding in MOD-m gates seems like a natural extension of AC0: for example, if m = 2, then we’re just adding Parity, one of the most basic functions not in AC0.
1124. u1124 Smolensky and Razborov extended the class of circuits for which lower bounds can be proven from AC0 to AC0 [p], whenever p is prime.
1125. u1125 Theorem 51 (Smolensky [243], Razborov [221]) Let p and q be distinct primes. Then Modq, the set of all strings with Hamming weight divisible by q, is not in AC0 [p].
1126. u1126 Indeed, any AC0 [p] circuit for Modq of depth d requires 2Ω(n1/2d) gates.
1127. u1127 As a corollary, the Majority function is also not in AC0 [p], and also requires 2Ω(n1/2d) gates to compute using AC0 [p] circuits of depth d.
1128. u1128 It’s not hard to show that AC0 [p] = AC0 [
1129. u1129 for any k ≥1, and thus, one also gets lower bounds against AC0 [m], whenever m is a prime power.
1130. u1130 The proof of Theorem 51 uses the so-called polynomial method.
1131. u1131 Here one argues that, if a function f can be computed by a constant-depth circuit with AND, OR, NOT, and MOD-p gates, then f can also be approximated by a low-degree polynomial over the finite field Fp.
1132. u1132 One then shows that a function of interest, such as the Modq function (for q̸ = p), can’t be approximated by any such low-degree polynomial. This provides the desired contradiction.
1133. u1133 The polynomial method is famously specific in scope: it’s still not known how to prove results like Theorem 51 even for AC0 [m] circuits, where m is not a prime power.49
1134. u1134 The reason why it breaks down there is simply that there are no finite fields of non-prime-power order. And thus, to be concrete, it’s still open whether the n-bit Majority function has a constant-depth, polynomialsize, unbounded-fanin circuit consisting of AND, OR, NOT, and MOD-6 gates, or even entirely of MOD-6 gates!
1135. u1135 Stepping back, it’s interesting to ask whether the constant-depth circuit lower bounds evade the relativization barrier explained in Section 6.1.2.
1136. u1136 There’s some disagreement about whether it’s even sensible to feed oracles to tiny complexity classes such as AC0 (see Allender and Gore [23] for example).
1137. u1137 However, to whatever extent it is sensible, the answer is that these lower bounds
1138. u1138 AC0)A, we mean AC0 extended by “oracle gates” that do evade relativization. For example, if by
1139. u1139 AC0)A = PA: for example, any A that is query A, then it’s easy to construct an A such that P-complete under AC0-reductions will work.
1140. u1140 On the other hand, we know from Theorem 50 that AC0̸ = P in the “real,” unrelativized world.
1141. u1141 There’s one other aspect of the constant-depth circuit lower bounds that requires discussion. Namely, we saw that both the random restriction method and the polynomial method yield lower bounds of the form 2nΩ(1/d) on the number of gates needed in an AC0 circuit of depth d—with the
1142. u1142 49Williams’s NEXP̸ ⊂ACC breakthrough [280], to be discussed in Section 6.4.2, could be seen as the first successful use of the “polynomial method” to prove a lower bound against AC0 [m] circuits—though in that case, the lower bound applies only to NEXP-complete problems, and polynomials are only one ingredient in the proof among many.
1143. u1143 sharpest result, due to H˚astad [123], having the form 2Ω(n1/(d−1)). Of course, this bound degrades rapidly as d gets large, and is never 2Ω(n) for any interesting depth d ≥3.
1144. u1144 So one might wonder:
1145. u1145 In the special case of Parity, why do these size lower bounds consistently fall short of 2Ω(n)?
1146. u1146 that question has a very simple answer: because 2O(n1/(d−1)) is easily seen to be a matching upper
1147. u1147 But could the same techniques yield a 2Ω(n) lower bound on AC0 circuit size for other bound!
1148. u1148 functions?
1149. u1149 The surprising answer is that 2nΩ(1/d) represents a sort of fundamental barrier—in the sense that, if we could do better, then we could also do much more than prove constant-depth circuit lower bounds.
1150. u1150 This is a consequence of the following result, which is not nearly as well-known as it should be: Theorem 52 (Allender et al. [25], Koiran [158]) Let L be a language decided by an NLOGSPACE (that is, nondeterministic logarithmic space) machine that halts in m = nO(1) time steps.
1151. u1151 Then for every odd constant d ≥3, we can also decide L using a family of AC0 circuits with depth d and size mO(m2/(d+1)).
1152. u1152 Theorem 52 means, in particular, that if we proved that some language L required AC0 circuits of size 2Ω(nε), independently of the constant depth d—even with, say, ε = 0.001—then we also would have proven L /∈NLOGSPACE. In other words, if we had the ability to prove strong enough AC0 lower bounds for languages in P, then we’d also have the ability to prove NLOGSPACE̸ = P.
1153. u1153 6.2.5
1154. u1154 The Natural Proofs Barrier Despite the weakness of AC0 and AC0 [p] circuits, the progress on lower bounds for them suggested what seemed to many researchers like a plausible path to proving NP̸ ⊂P/poly, and hence P̸ = NP. That path is simply to generalize the random restriction and polynomial methods further and further, to get lower bounds for more and more powerful classes of circuits.
1155. u1155 The first step, of course, would be to generalize the polynomial method to handle AC0 [m] circuits, where m is not
1156. u1156 Then one could handle what are called TC0 circuits: that is, constant-depth, a prime power.
1157. u1157 polynomial-size, unbounded-fanin circuits with MAJORITY gates (or, as in a neural network, threshold gates, which output 1 if a certain weighted affine combination of the input bits exceeds 0, and 0 otherwise).
1158. u1158 Next, one could aim for polynomial-size circuits of logarithmic depth: that is, the class NC1. Finally, one could push all the way to polynomial-depth circuits: that is, the class P/poly.
1159. u1159 Unfortunately, we now know that this path hits a profound barrier at TC0, if not earlier—a barrier that explains why the random restriction and polynomial methods haven’t taken us further toward a proof of P̸ = NP.
1160. u1160 Apparently this barrier was known to Michael Sipser (and perhaps a few others) in the 1980s, but it was first articulated in print in 1993 by Razborov and Rudich [223], who called it the natural proofs barrier.
1161. u1161 The basic insight is that combinatorial techniques, such as the method of random restrictions, do more than advertised: in some sense, they do too much for their own good.
1162. u1162 In particular, not only do they let us show that certain specific functions, like Parity, are hard for AC0; they even let us certify that a random function is hard for AC0.
1163. u1163 Indeed, such techniques give rise to an algorithm, which takes as input the truth table of a Boolean function f : {0, 1}n →{0, 1}, and which has the following two properties.
1164. u1164 (1) “Constructivity.”
1165. u1165 The algorithm runs in time polynomial in the size of f’s truth table
1166. u1166 If f is chosen uniformly at random, then with probability at least 1/nO(1)
1167. u1167 (2) “Largeness.”
1168. u1168 over f, the algorithm certifies that f is hard (i.e., that f is not in some circuit class C, such
1169. u1169 as AC0 in the case of the random restriction method).
1170. u1170 If a lower bound proof gives rise to an algorithm satisfying (1) and (2), then Razborov and Rudich call it a natural proof.
1171. u1171 In many cases, it’s not entirely obvious that a lower bound proof is natural, but with some work one can show that it is.
1172. u1172 To illustrate, in the case of the random restriction method, the algorithm could check that f has a large fraction of its Fourier mass on high-degree Fourier coefficients, or that f has high “average sensitivity” (that is, if x and y are random inputs that differ only in a single bit, then with high probability f (x)̸ = f (y)).
1173. u1173 These tests have the following three properties:
1174. u1174 They’re easy to perform, in time polynomial in the truth table size 2n.
1175. u1175 A random function f will pass these tests with overwhelming probability (that is, such an f
1176. u1176 will “look like Parity” in the relevant respects).
1177. u1177 The results of Linial, Mansour, and Nisan [171] show that any f that passes these tests
1178. u1178 remains nontrivial under most random restrictions, and for that reason, can’t be in AC0.
1179. u1179 But now, twisting the knife, Razborov and Rudich point out that any natural lower bound proof against a powerful enough complexity class would be self-defeating, in that it would yield an efficient algorithm to solve some of the same problems that we’d set out to prove were hard. More concretely, suppose we have a natural lower bound proof against the circuit class C.
1180. u1180 Then by definition, we also have an efficient algorithm A that, given a random Boolean function f : {0, 1}n →{0, 1}, certifies that f /∈C with at least 1/nO(1) probability over f. But this means that C cannot contain very strong families of pseudorandom functions: namely, functions f : {0, 1}n →{0, 1} that are indistinguishable from “truly” random functions, even by algorithms that can examine their entire truth tables and use time polynomial in 2n.
1181. u1181 Because A can never certify f /∈C if f is a pseudorandom function, computable
1182. u1182 But A certifies f /∈C with 1/nO(1) probability over a truly random f.
1183. u1183 in C.
1184. u1184 Thus, A serves to distinguish random from pseudorandom functions with non-negligible50 bias—so the latter were never really pseudorandom at all.
1185. u1185 To recap, we’ve shown that, if there’s any natural proof that any function is not in C, then all Boolean functions computable in C can be distinguished from random functions by 2O(n)-time algorithms. That might not sound so impressive, since 2O(n) is a lot of time. But a key observation is that, for most of the circuit classes C that we care about, there are families of pseudorandom functions {fs}s on n bits that are conjectured to require 2p(n) time to distinguish from truly random functions, where p (n) is as large a polynomial as we like (related to the length of the random “seed” s). It follows from results of Naor and Reingold [200] that in TC0 (constant-depth, polynomial-size threshold circuits), there are functions that can’t be distinguished from random ( functions in 2O(n) time, unless the factoring and discrete logarithm problems are solvable in O
1186. u1186 time for every
1187. u1187 50In theoretical computer science, the term non-negligible means lower-bounded by 1/nO(1).
1188. u1188 (For comparison, the best known algorithms for these problems take roughly 2n1/3 time.) ε > 0.
1189. u1189 Likewise, Banerjee et al. [39] showed that in TC0, there are functions that can’t be distinguished ( from random in 2O(n) time, unless noisy systems of linear equations can be solved in O
1190. u1190 time for every ε > 0.
1191. u1191 It’s worth pausing to let the irony sink in. Razborov and Rudich are pointing out that, as we showed certain problems (factoring and discrete logarithm) to be harder and harder via a natural proof, we’d simultaneously show those same problems to be easier and easier! Indeed, any natural proof showing that these problems took at least t (n) time, would also show that they took at most roughly 2t−1(n) time. As a result, no natural proof could possibly show these problems take more than half-exponential time: that is, time t (n) such that t (t (n)) grows exponentially.
1192. u1192 ?= NP
1193. u1193 Here, perhaps, we’re finally face-to-face with a central conceptual difficulty of the P question: namely, we’re trying to prove that certain functions are hard, but the problem of deciding whether a function is hard is itself hard, according to the very sorts of conjectures that we’re trying to prove.51
1194. u1194 Of course, the natural proofs barrier didn’t prevent complexity theorists from proving strong lower bounds against AC0.
1195. u1195 But the result of Linial, Mansour, and Nisan [171] can be interpreted as saying that this is because AC0 is not yet powerful enough to express pseudorandom functions.52 When we move just slightly higher, to TC0 (constant-depth threshold circuits), we do have pseudorandom functions under plausible hardness assumptions, and—not at all coincidentally, according to Razborov and Rudich—we no longer have strong circuit lower bounds.
1196. u1196 In that sense, natural proofs explains almost precisely why the progress toward proving P̸ = NP via circuit complexity
1197. u1197 The one complication in the story is the AC0 [m] classes, for which we don’t stalled where it did.
1198. u1198 yet have strong lower bounds (though see Section 6.4), but also don’t have pseudorandom function candidates. For those classes, it’s still possible that natural proofs could succeed.
1199. u1199 As Razborov and Rudich themselves stressed, the take-home message is not that we should give up on proving P̸ = NP.
1200. u1200 In fact, since the beginning of complexity theory, we’ve had at least one technique that easily evades the natural proofs barrier: namely, diagonalization (the technique used to prove P̸ = EXP; see Section 6.1)!
1201. u1201 The reason why diagonalization evades the barrier is that it zeroes in on a specific property of the function f being lower-bounded—namely, the fact that f is EXP-complete, and thus able to simulate all P machines—and thereby avoids the trap of arguing that “f is hard because it looks like a random function.” Of course, diagonalization is subject to the relativization barrier (see Section 6.1.2), so the question still stands of how to evade relativization and natural proofs simultaneously; we’ll return to that question in Section 6.3.
1202. u1202 More broadly, there are many cases in mathematics where we can prove that some object O of interest to us has a property P, even though we have no hope of finding a general polynomialtime algorithm to decide whether any given object has property P, or even to certify a large fraction of objects as having property P. In such cases, often we prove that O has property P by
1203. u1203 51Technically, the problem of distinguishing random from pseudorandom functions is equivalent to the problem of inverting one-way functions, which is not quite as strong as solving NP-complete problems in polynomial time—only solving average-case NP-complete problems with planted solutions. For more see Section 5.3.
1204. u1204 52Though even here, Theorem 52 implies that for every ε > 0 and c, there exists a d such that AC0 circuits of size 2nε 52Though and depth even d can here, simulate Theorem an 52 NLOGSPACE implies that machine for every that ε > 0 halts and in c, there time nc.
1205. u1205 But it’s known that NLOGSPACE can simulate uniform NC1, which contains strong pseudorandom function candidates. So even for AC0 circuits, if we wanted to prove strong enough exponential lower bounds on size, then we would be up against the natural proofs
1206. u1206 Not by coincidence, proving such strong size lower bounds for AC0 has been a longstanding open problem, barrier.
1207. u1207 as discussed in Section 6.2.4.
1208. u1208 exploiting special symmetries in O—symmetries that have little to do with why O has property P, but everything to do with why we can prove it has the property. As an example, a random graph is an expander graph (that is, a graph on which a random walk mixes rapidly) with overwhelming probability. But since the general problem of deciding whether a graph is an expander is NP-hard, if we want a specific graph G that’s provably an expander, typically we need to construct G with a large amount of symmetry: for example, by taking it to be the Cayley graph of a finite group. Similarly, even though we expect that there’s no general efficient algorithm to decide if a Boolean function f is hard,53 given as input f’s truth table, we might be able to prove that certain specific f’s (for example, NP- or #P-complete ones) are hard by exploiting their symmetries.
1209. u1209 Geometric Complexity Theory (see Section 6.6) is the best-known development of that particular hope for escaping the natural proofs barrier.
1210. u1210 But GCT is not the only way to use symmetry to evade natural proofs.
1211. u1211 As a vastly smaller example, I [2, Appendix 10] proved an exponential lower bound on the so-called manifestly orthogonal formula size of a function f : {0, 1}n →{0, 1} that outputs 1 if the input x is a codeword of a linear error-correcting code, and 0 otherwise. Here a manifestly orthogonal formula is a formula over x1, . . . , xn, x1, . . . , xn consisting of OR and AND gates, where every OR must be of two subformulas over the same set of variables, and every AND must be of two subformulas over disjoint sets of variables.
1212. u1212 My lower bound wasn’t especially difficult, but what’s notable about it is that it took crucial advantage of a symmetry of linear error-correcting codes: namely, the fact that any such code can be recursively decomposed as a disjoint union of Cartesian products of smaller linear error-correcting codes. My proof thus gives no apparent insight into how to certify that a random Boolean function has manifestly orthogonal formula size exp (n), and possibly evades the natural proofs barrier (if there is such a barrier in the first place for manifestly orthogonal formulas).
1213. u1213 Another proposal for how to evade the natural proofs barrier comes from a beautiful 2010 paper by Allender and Kouck´y [26] (see also Allender’s survey [21]).
1214. u1214 These authors show that, if one wanted to prove that certain specific NC1 problems were not in TC0, thereby establishing the breakthrough separation TC0̸ = NC1, it would suffice to show that those problems had no TC0 circuits of size n1+ε, for any constant ε > 0.
1215. u1215 To achieve this striking “bootstrap,” from an n1+ε lower bound to a superpolynomial one, Allender and Kouck´y exploit the self-reducibility of the NC1 problems in question: the fact that they can be reduced to smaller instances of themselves. Crucially, this self-reducibility would not hold for a random function. For this reason, the proposed lower bound method has at least the potential to evade the natural proofs barrier.
1216. u1216 Indeed, it’s not even totally implausible that a natural proof could yield an n1+ε lower bound for TC0 circuits, with the known bootstrapping from n1+ε to superpolynomial the only non-natural part of the argument.54
1217. u1217 53The problem, given as input the truth table of a Boolean function f : {0, 1}n →{0, 1}, of computing or approximating the circuit complexity of f is called the Minimum Circuit Size Problem (MCSP). It’s a longstanding open problem whether or not MCSP is NP-hard; at any rate, there are major obstructions to proving it NP-hard with existing techniques (see Kabanets and Cai [139] and Murray and Williams [199]).
1218. u1218 On the other hand, MCSP can’t be in P (or BPP) unless there are no cryptographically-secure pseudorandom generators.
1219. u1219 At any rate, what’s relevant to natural proofs is just whether there’s an efficient algorithm to certify a large fraction of Boolean functions as being hard: that is, to output “f is hard” for a 1/ poly (n) fraction of Boolean functions f : {0, 1}n →{0, 1} that require large circuits, and for no Boolean functions that have small circuits. This is a weaker requirement than solving MCSP.
1220. u1220 54Allender and Kouck´y’s paper partly builds on 2003 work by Srinivasan [246], who showed that, to prove P̸ = NP, one would ( “merely” need to show that any algorithm to compute weak approximations for the MaxClique problem takes Ω
1221. u1221 The way Srinivasan proved this striking statement was, again, by
1222. u1222 I can’t resist mentioning a final idea about how to evade natural proofs.
1223. u1223 In a 2014 paper, Chapman and Williams [72] suggested proving circuit lower bounds for NP-complete problems like 3Sat, via arguments that would work only for circuits that are self-certifying: that is, that output satisfying assignments whenever they exist, which we know that a circuit solving NP-complete problems can always do by Proposition 4.
1224. u1224 Strikingly, they then showed that if NP̸ ⊂P/poly is true at all, then it has a proof that’s “natural” in their modified sense: that is, a proof that yields an efficient algorithm to certify that a 1/nO(1) fraction of all Boolean functions f : {0, 1}n →{0, 1} either don’t have polynomial-size circuits or else aren’t self-certifying. Thus, if we could just figure out how to exploit NP-complete problems’ property of self-certification, that would already be enough to evade natural proofs.
1225. u1225 6.3
1226. u1226 Arithmetization In the previous sections, we saw that there are logic-based techniques (like diagonalization) that suffice to prove P̸ = EXP and NEXPNP̸ ⊂P/poly, and that evade the natural proofs barrier, but that are blocked from proving P̸ = NP by the relativization barrier. Meanwhile, there are combinatorial techniques (like random restrictions) that suffice to prove circuit lower bounds against AC0 and AC0 [p], and that evade the relativization barrier, but that are blocked from proving lower bounds against P/poly (and hence, from proving P̸ = NP) by the natural proofs barrier.
1227. u1227 This raises a question: couldn’t we simply combine techniques that evade relativization but not natural proofs, with techniques that evade natural proofs but not relativization, in order to evade both? As it turns out, we can.
1228. u1228 6.3.1 The story starts with a dramatic development in complexity theory around 1990, though not one that obviously bore on P̸ = NP or circuit lower bounds.
1229. u1229 In the 1980s, theoretical cryptographers became interested in so-called interactive proof systems, which are protocols where a computationally-unbounded but untrustworthy prover (traditionally named Merlin) tries to convince a skeptical polynomial-time verifier (traditionally named Arthur) that some mathematical statement is true, via a two-way conversation, in which Arthur can randomly generate challenges and then evaluate Merlin’s answers to them.
1230. u1230 More formally, let IP (Interactive Proof) be the class of all languages L ⊆{0, 1}∗for which there exists a probabilistic polynomial-time algorithm for Arthur with the following properties. Arthur receives an input string x ∈{0, 1}n (which Merlin also knows), and then generates up to nO(1)
1231. u1231 Each challenge is a string of up to nO(1) bits, and each can depend challenges to send to Merlin.
1232. u1232 on x, on Arthur’s internal random bits, and on Merlin’s responses to the previous challenges. (We also allow Arthur, if he likes, to keep some random bits hidden, without sending them to Merlin— though surprisingly, this turns out not to make any difference [109].) We think of Merlin as trying his best to persuade Arthur that x ∈L; at the end, Arthur decides whether to accept or reject Merlin’s claim. We require that for all inputs x:
1233. u1233 If x ∈L, then there’s some strategy for Merlin (i.e., some function determining which message using a sort of self-reduciblity: he showed that, if there’s a polynomial-time algorithm for MaxClique, then by running that algorithm on smaller graphs sampled from the original graph, one can solve approximate versions of MaxClique in n1+o(1) time.
1234. u1234 to send next, given x and the sequence of challenges so far55) that causes Arthur to accept
1235. u1235 with probability at least 2/3 over his internal randomness.
1236. u1236 If x /∈L, then regardless of what strategy Merlin uses, Arthur rejects with probability at least
1237. u1237 Clearly IP generalizes NP: indeed, we recover NP if we get rid of the interaction and randomness aspects, and just allow a single message from Merlin, which Arthur either accepts or rejects.
1238. u1238 In the other direction, it’s not hard to show that IP ⊆PSPACE.56
1239. u1239 The question asked in the 1980s was: does interaction help?
1240. u1240 In other words, how much bigger is IP than NP?
1241. u1241 It was observed that IP contains at least a few languages that aren’t known to be in NP, such as graph non-isomorphism.
1242. u1242 This is so because of a simple, famous, and elegant protocol [108]: given two n-vertex graphs G and H, Arthur can pick one of the two uniformly at random, randomly permute its vertices, then send the result to Merlin. He then challenges Merlin: which graph did I start from, G or H?
1243. u1243 If G ≇H, then Merlin, being computationally unbounded,
1244. u1244 If, on the other hand, G ∼= H, can easily answer this challenge by solving graph isomorphism.
1245. u1245 then Merlin sees the same distribution over graphs regardless of whether Arthur started from G or H, so he must guess wrongly with probability 1/2.
1246. u1246 Despite such protocols, the feeling in the late 1980s was that IP should be only a “slight” extension of NP.
1247. u1247 This feeling was buttressed by a result of Fortnow and Sipser [97], which said that there exists an oracle A such that coNPA̸ ⊂IPA, and hence, any interactive protocol even for coNP (e.g., for proving Boolean formulas unsatisfiable) would require non-relativizing techniques.
1248. u1248 Yet in the teeth of that oracle result, Lund, Fortnow, Karloff, and Nisan [178] showed nevertheless that coNP ⊆IP “in the real world”—and not only that, but P#P ⊆IP.
1249. u1249 This was quickly improved by Shamir [234] to the following striking statement: Theorem 53 ([178, 234]) IP = PSPACE.
1250. u1250 Theorem 53 means, for example, that if a computationally-unbounded alien came to Earth, it could not merely beat us in games of strategy like chess: rather, the alien could mathematically prove to us, via a short conversation and to statistical certainty, that it knew how to play perfect chess.57
1251. u1251 Theorem 53 has been hugely influential in complexity theory for several reasons, but one reason was that it illustrated, dramatically and indisputably, that the relativization barrier need not inhibit progress.
1252. u1252 So how was this amazing result achieved, and why does the proof fail relative to certain oracles? The trick is what we now call arithmetization.
1253. u1253 This means that we take a Boolean formula or circuit—involving, for example, AND, OR, and NOT gates—and then reinterpret the Boolean gates as arithmetic operations over some larger finite field Fp. More concretely, the Boolean AND (x∧y)
1254. u1254 55We can assume without loss of generality that Merlin’s strategy is deterministic, since Merlin is computationally unbounded, and any convex combination of strategies must contain a deterministic strategy that causes Arthur to accept with at least as great a probability as the convex combination does.
1255. u1255 56This is so because a polynomial-space Turing machine can treat the entire interaction between Merlin and Arthur as a game, in which Merlin is trying to get Arthur to accept with the largest possible probability. The machine can then evaluate the exponentially large game tree using depth-first recursion.
1256. u1256 57This assumes that we restrict attention to n × n chess games that end after at most nO(1) moves, for example because of the time limits in tournament play.
1257. u1257 As discussed in Section 2.2.5, with no time limits chess jumps up from being PSPACE-complete to being EXP-complete.
1258. u1258 becomes multiplication (xy), the Boolean NOT becomes the function 1 −x, and the Boolean OR (x ∨y) becomes x + y −xy.
1259. u1259 Note that if x, y ∈{0, 1}, then we recover the original Boolean operations.
1260. u1260 But the new operations make sense even if x, y /∈{0, 1}, and they have the effect of lifting our Boolean formula or circuit to a multivariate polynomial over Fp.
1261. u1261 Furthermore, the degree of the polynomial can be upper-bounded in terms of the size of the formula or circuit.
1262. u1262 The advantage of this lifting is that polynomials, at least over large finite fields, have powerful error-correcting properties that are unavailable in the Boolean case.
1263. u1263 These properties ultimately derive from a basic fact of algebra: a nonzero, degree-d univariate polynomial has at most d roots. As a consequence, if q, q′ : Fp →Fp are two degree-d polynomials that are unequal (and d ≪p), then with high probability, their inequality can be seen by querying them at a random point:
1264. u1264 Let me now give a brief impression of how one proves Theorem 53, or at least the simpler result
1265. u1265 Let φ (x1, . . . , xn) be, say, a 3Sat formula that Merlin wants to convince Arthur is coNP ⊆IP.
1266. u1266 unsatisfiable. Then Arthur first lifts φ to a multivariate polynomial q : Fn p
1267. u1267 p →Fp, of degree d ≤|φ| (where |φ| is the size of φ), over the finite field Fp, for some p ≫2n. Merlin’s task is equivalent to convincing Arthur of the following equation:
1268. u1268 To achieve this, Merlin first sends Arthur the coefficients of a univariate polynomial q1 : Fp →Fp. Merlin claims that q1 satisfies
1269. u1269 (1)
1270. u1270 and also satisfies q1 (0) + q1 (1) = 0.
1271. u1271 Arthur can easily check the latter equation for himself.
1272. u1272 To check equation (1), Arthur picks a random value r1 ∈Fp for x1 and sends it to Merlin.
1273. u1273 Then Merlin replies with a univariate polynomial q2, for which he claims that
1274. u1274 Arthur checks that q2 (0)+q2 (1) = q1 (r1), then picks a random value r2 ∈Fp for x2 and sends it to Merlin, and so on. Finally, Arthur checks that qn is indeed the univariate polynomial obtained by starting from the arithmetization of φ, then fixing x1, . . . , xn−1 to r1, . . . , rn−1 respectively.
1275. u1275 The bounded number of roots ensures that, if Merlin lied at any point in the protocol, then with high probability at least one of Arthur’s checks will fail.
1276. u1276 Now, to return to the question that interests us: why does this protocol escape the relativization barrier?
1277. u1277 The short answer is: because if the Boolean formula φ involved oracle gates, then we wouldn’t have been able to arithmetize φ.
1278. u1278 By arithmetizing φ, we did something “deeper” with it, more dependent on its structure, than simply evaluating φ on various Boolean inputs (which would have continued to work fine had an oracle been involved).
1279. u1279 Arithmetization made sense because φ was built out of AND and OR and NOT gates, which we were able to reinterpret arithmetically. But how would we arithmetically reinterpret an oracle gate?
1280. u1280 6.3.2
1281. u1281 Hybrid Circuit Lower Bounds To recap, PSPACE ⊆IP is a non-relativizing inclusion of complexity classes.
1282. u1282 But can we leverage that achievement to prove non-relativizing separations between complexity classes, with an eye toward P̸ = NP? ( Certainly, by combining IP = PSPACE with the Space Hierarchy ( Theorem (which implies SPACE
1283. u1283 = PSPACE for every fixed k), we get that IP̸ ⊂SPACE
1284. u1284 for every fixed k. Likewise, by combining IP = PSPACE with Theorem 41 (that PSPACE does not have circuits of size nk for fixed k), we get that IP doesn’t have circuits of size nk either.
1285. u1285 Furthermore, both of these separations can be shown to be non-relativizing, using techniques from [62]. But can we get more interesting separations?
1286. u1286 The key to doing so turns out to be a beautiful corollary of the IP = PSPACE theorem.
1287. u1287 To state the corollary, we need one more complexity class: MA (Merlin-Arthur) is a probabilistic
1288. u1288 It’s defined as the class of languages L ⊆{0, 1}∗for which there exists a generalization of NP.
1289. u1289 probabilistic polynomial-time verifier M, and a polynomial p, such that for all inputs x ∈{0, 1}∗:
1290. u1290 If x ∈L then there exists a witness string w ∈{0, 1}p(|x|) such that M (x, w) accepts with
1291. u1291 probability at least 2/3 over its internal randomness.
1292. u1292 If x /∈L, then M (x, w) rejects with probability at least 2/3 over its internal randomness, for
1293. u1293 Clearly MA contains NP and BPP. It can also be shown that MA ⊆ΣP 2
1294. u1294 2 and that MA ⊆PP, where PP is the counting class from Section 2.2.6. Now, here’s the corollary of Theorem 53: Corollary 54 If PSPACE ⊂P/poly, then PSPACE = MA.
1295. u1295 Proof. Suppose PSPACE ⊂P/poly, let L ∈PSPACE, and let x be an input in L. Then as an MA witness proving that x ∈L, Merlin simply sends Arthur a description of a polynomial-size circuit C that simulates the PSPACE prover, in an interactive protocol that convinces Arthur that x ∈L. (Here we use one additional fact about Theorem 53, beyond the mere fact that IP = PSPACE: that, in the protocol, Merlin can run a PSPACE algorithm to decide which message to send next.) Then Arthur simulates the protocol, using C to compute Merlin’s responses to his random challenges, and accepts if and only if the protocol does. Hence L ∈MA.
1296. u1296 Corollary 55 If P#P ⊂P/poly, then P#P = MA.
1297. u1297 (Again, here we use the observation that, in the protocol proving that P#P ⊆IP, Merlin can run a P#P algorithm to decide which message to send next.)
1298. u1298 Let’s now see how we can use these corollaries of IP = PSPACE to prove new circuit lower
1299. u1299 Let MAEXP be “the exponential-time version of MA,” with a 2p(n)-size witness that can bounds.
1300. u1300 be probabilistically verified in 2p(n) time: in other words, the class that is to MA as NEXP is to NP. Then: Theorem 56 (Buhrman-Fortnow-Thierauf [62]) MAEXP̸ ⊂P/poly.
1301. u1301 Proof. Suppose by contradiction that MAEXP ⊂P/poly. Then certainly PSPACE ⊂P/poly, which means that PSPACE = MA by Corollary 54.
1302. u1302 By a padding argument (see Proposition 17), this means that EXPSPACE = MAEXP.
1303. u1303 But we already saw in Theorem 40 that EXPSPACE̸ ⊂P/poly, and therefore MAEXP̸ ⊂P/poly as well.
1304. u1304 Note in particular that if we could prove MA = NP, then we’d also have MAEXP = NEXP by padding, and hence NEXP̸ ⊂P/poly by Theorem 56.
1305. u1305 This provides another example of how derandomization can lead to circuit lower bounds, a theme mentioned in Section 6.
1306. u1306 A second example involves the class PP.
1307. u1307 Theorem 57 (Vinodchandran [267]) For every fixed k, there is a language in PP that does not have circuits of size nk.
1308. u1308 Proof. Fix k, and suppose by contradiction that PP has circuits of size nk.
1309. u1309 Then in particular, PP ⊂P/poly, so PPP = P#P ⊂P/poly, so P#P = PP = MA by Corollary 55.
1310. u1310 But we noted in Section 6.1 that ΣP 2
1311. u1311 2 does not have circuits of size nk. And ΣP 2
1312. u1312 2 ⊆P#P by Toda’s Theorem (Theorem 13), so P#P doesn’t 2 have circuits of size nk either. Therefore 2 neither does PP.58
1313. u1313 As a final example, Santhanam [228] showed the following (we omit the proof). Theorem 58 (Santhanam [228]) For every fixed k, there is an MA “promise problem”59 that does not have circuits of size nk.
1314. u1314 The above results clearly evade the natural proofs barrier, because they give lower bounds against strong circuit classes such as P/poly, or the set of all size-nk circuits for fixed k.
1315. u1315 This is not so surprising when we observe that the proofs build on the simpler results from Section 6.1, which already used diagonalization to evade the natural proofs barrier.
1316. u1316 What’s more interesting is that these results also evade the relativization barrier.
1317. u1317 Of course, one might guess as much, after noticing that the proofs use the non-relativizing IP = PSPACE theorem.
1318. u1318 But to show rigorously that the circuit lower bounds themselves fail to relativize, one needs to construct oracles relative to which the circuit lower bounds are false. This is done by the following results, whose somewhat elaborate proofs we omit: Theorem 59 (Buhrman-Fortnow-Thierauf [62]) There exists an oracle A such that MAA
1319. u1319 EXP ⊂ PA/poly.
1320. u1320 Theorem 60 (Aaronson [4]) There exists an oracle A relative to which all languages in PP have linear-sized circuits.
1321. u1321 58Actually, for this proof one does not really need either Toda’s Theorem, or the slightly-nontrivial result that ΣP 2
1322. u1322 2 does not have circuits of size nk.
1323. u1323 Instead, one can just argue directly that at any rate, P#P does not have circuits 2 of size nk, using a slightly more careful version of the argument of Theorem 40. For details see Aaronson [4].
1324. u1324 59In complexity theory, a promise problem is a pair of subsets ΠYES, ΠNO ⊆{0, 1}∗with ΠYES ∩ΠNO = ∅.
1325. u1325 An algorithm solves the problem if it accepts all inputs in ΠYES and rejects all inputs in ΠNO.
1326. u1326 Its behavior on inputs neither in ΠYES nor ΠNO (i.e., inputs that “violate the promise”) can be arbitrary.
1327. u1327 A typical example of a promise problem is: given a Boolean circuit C, decide whether C accepts at least 2/3 of all inputs x ∈{0, 1}n or at most 1/3 of them, promised that one of those is true.
1328. u1328 This problem is in BPP (or technically, PromiseBPP).
1329. u1329 The role of the promise here is to get rid of those inputs for which random sampling would accept with probability between 1/3 and 2/3, violating the definition of BPP.
1330. u1330 The proofs of both of these results also easily imply that there exists an oracle relative to which all MA promise problems have linear-sized circuits.
1331. u1331 The bottom line is that, by combining non-relativizing results like IP = PSPACE with nonnaturalizing results like EXPSPACE̸ ⊂P/poly, we can prove interesting circuit lower bounds that neither relativize nor naturalize. So then why couldn’t we keep going, and use similar techniques to prove NEXP̸ ⊂P/poly, or even P̸ = NP? Is there a third barrier, to which even the arithmetizationbased lower bounds are subject?
1332. u1332 6.3.3
1333. u1333 The Algebrization Barrier In 2008, Avi Wigderson and I [10] showed that, alas, there’s a third barrier. In particular, while the arithmetic techniques used to prove IP = PSPACE do evade relativization, they crash up against a modified version of relativization that’s “wise” to those techniques.
1334. u1334 We called this modified barrier the algebraic relativization or algebrization barrier. We then showed that, in order to prove P̸ = NP—or for that matter, even to prove NEXP̸ ⊂P/poly, or otherwise go even slightly beyond the results of Section 6.3.2—we’d need techniques that evade the algebrization barrier (and also, of course, evade natural proofs).
1335. u1335 In more detail, we can think of an oracle as just an infinite collection of Boolean functions, fn : {0, 1}n →{0, 1} for each n.
1336. u1336 Now, by an algebraic oracle, we mean an oracle that provides access not only to fn for each n, but also to a low-degree extension fn f
1337. u1337 fn f : Fn →F of fn over some large finite field F. This extension must have the property that fn f
1338. u1338 fn f (x) = fn (x) for all x ∈{0, 1}n, and it must be a polynomial of low degree—say, at most 2n.
1339. u1339 But such extensions always exist,60 and querying them outside the Boolean cube {0, 1}n might help even for learning about the Boolean part fn.
1340. u1340 The point of algebraic oracles is that they capture what we could do if we had a formula or circuit for fn, and were willing to evaluate it not only on Boolean inputs, but on non-Boolean ones as well, in the manner of IP = PSPACE.
1341. u1341 In particular, we saw in Section 6.3.1 that, given (say) a 3Sat formula φ, we can “lift” φ to a low-degree polynomial eφ over a finite field F by reinterpreting the AND, OR, and NOT gates in terms of field addition and multiplication.
1342. u1342 So if we’re trying to capture the power of arithmetization relative to an oracle function fn, then it stands to reason that we should also be allowed to lift fn.
1343. u1343 Once we do so, we find that the non-relativizing results based on arithmetization, such as IP = PSPACE, relativize with respect to algebraic oracles (or “algebrize”). That is:
1344. u1344 A e for all algebraic oracles eA. Likewise, PSPACE
1345. u1345 Theorem 61 IP
1346. u1346 A/poly e implies
1347. u1347 A e for all algebraic oracles eA, and so on for all the interactive proof results. PSPACE
1348. u1348 The intuitive reason is that, any time (say) Arthur needs to arithmetize a formula φ containing A-oracle gates in an interactive protocol, he can handle non-Boolean inputs to the A-oracle gates by calling eA.
1349. u1349 As a consequence of Theorem 61, the circuit lower bounds of Section 6.3.2 are algebrizing as
1350. u1350 A e does not have well: for example, for all algebraic oracles eA, we have MA size-nk circuits with eA-oracle gates.
1351. u1351 60Indeed, every fn has an extension to a degree-n polynomial, namely a multilinear one (in which no variable is raised to a higher power than 1): for example, OR (x, y) = x + y −xy.
1352. u1352 Admittedly, the original paper of Aaronson and Wigderson [10] only managed to prove a weaker version of Theorem 61. It showed, for example, that for all algebraic oracles eA, we have PSPACEA ⊆ IP
1353. u1353 A, e and MA
1354. u1354 A EXP̸ e ⊂PA/poly. As a result, it had to define algebrization in a convoluted way, where some complexity classes received the algebraic oracle eA while others received only the “original” oracle A, and which class received which depended on what kind of result one was talking about (e.g., an inclusion or a separation).
1355. u1355 Shortly afterward, Impagliazzo, Kabanets, and Kolokolova [133] fixed this defect of algebrization, proving Theorem 61 even when all classes receive the same algebraic oracle eA, but only at the cost of jettisoning Aaronson and Wigderson’s conclusion that any proof of NEXP̸ ⊂P/poly will require non-algebrizing techniques.
1356. u1356 Very recently, Aydınlıo˘glu and Bach [34] showed how to get the best of both worlds, with a uniform definition of algebrization and the conclusion about NEXP vs. P/poly.
1357. u1357 In any case, the main point of [10] was that to prove P̸ = NP, or otherwise go further than the circuit lower bounds of Section 6.3.2, we’ll need non-algebrizing techniques: techniques that fail to relativize in a “deeper” way than IP = PSPACE fails to relativize.
1358. u1358 Let’s see why this is true for P̸ = NP.
1359. u1359 Theorem 62 (Aaronson-Wigderson [10]) There exists an algebraic oracle eA such that P A e
1360. u1360 A e = NP
1361. u1361 As a consequence, any proof of P̸ = NP will require non-algebrizing techniques. Proof. We can just let A be any PSPACE-complete language, and then let eA be its unique extension to a collection of multilinear polynomials over F (that is, polynomials in which no variable is ever raised to a higher power than 1).
1362. u1362 The key observation is that the multilinear extensions are themselves computable in PSPACE. So we get a PSPACE-complete oracle eA, which collapses P and NP for the same reason as in the original argument of Baker, Gill, and Solovay [38] (see Theorem 43).
1363. u1363 Likewise, Aaronson and Wigderson [10] showed that any proof of P = NP, or even P = BPP, would need non-algebrizing techniques.
1364. u1364 They also proved the following somewhat harder result, whose proof we omit.
1365. u1365 Theorem 63 ([10]) There exists an algebraic oracle eA such that NEXP
1366. u1366 As a consequence, any proof of NEXP̸ ⊂P/poly will require non-algebrizing techniques.
1367. u1367 Note that this explains almost exactly why progress stopped where it did: MAEXP̸ ⊂P/poly can be proved with algebrizing techniques, but NEXP̸ ⊂P/poly can’t be.
1368. u1368 I should mention that Impagliazzo, Kabanets, and Kolokolova [133] gave a logical interpretation of algebrization, extending the logical interpretation of relativization given by Arora, Impagliazzo, and Vazirani [30]. Impagliazzo et al. show that the algebrizing statements can be seen as all those statements that follow from “algebrizing axioms for computation,” which include basic closure properties, and also the ability to lift any Boolean computation to a larger finite field. Statements like P̸ = NP are then provably independent of the algebrizing axioms.
1369. u1369 6.4
1370. u1370 Ironic Complexity Theory There’s one technique that’s had some striking recent successes in proving circuit lower bounds, and that bypasses the natural proofs, relativization, and algebrization barriers.
1371. u1371 This technique might be called “ironic complexity theory.” It uses the existence of surprising algorithms in one setting
1372. u1372 to show the nonexistence of algorithms in another setting.
1373. u1373 It thus reveals a “duality” between upper and lower bounds, and reduces the problem of proving impossibility theorems to the much better-understood task of designing efficient algorithms.61
1374. u1374 At a conceptual level, it’s not hard to see how algorithms can lead to lower bounds.
1375. u1375 For example, suppose someone discovered a way to verify arbitrary exponential-time computations efficiently, thereby proving NP = EXP. Then as an immediate consequence of the Time Hierarchy Theorem (P̸ = EXP), we’d get P̸ = NP.
1376. u1376 Or suppose someone discovered that every language in P had linear-size circuits.
1377. u1377 Then P = NP would imply that every language in PH had linear-size circuits—but since we know that’s not the case (see Section 6.1), we could again conclude that P̸ = NP.
1378. u1378 Conversely, if someone proved P = NP, that wouldn’t be a total disaster for lower bounds research: at least it would immediately imply EXP̸ ⊂P/poly (via EXP = EXPNPNP), and the existence of languages in P and NP that don’t have linear-size circuits!
1379. u1379 Examples like this can be multiplied, but there’s an obvious problem with them: they each show a separation, but only assuming a collapse that’s considered extremely unlikely to happen. However, recently researchers have managed to use surprising algorithms that do exist, and collapses that do happen, to achieve new lower bounds. In this section I’ll give two examples.
1380. u1380 6.4.1
1381. u1381 Time-Space Tradeoffs At the moment, no one can prove that solving 3Sat requires more than linear time (let alone exponential time!), on realistic models of computation like random-access machines.62
1382. u1382 Nor can anyone prove that solving 3Sat requires more than O (log n) bits of memory.
1383. u1383 But the situation isn’t completely hopeless: at least we can prove there’s no algorithm for 3Sat that uses both linear time and logarithmic memory! Indeed, we can do better than that.
1384. u1384 A “time-space tradeofftheorem” shows that any algorithm to solve some problem must use
1385. u1385 The first such theorem for 3Sat was proved either more than T time or else more than S space.
1386. u1386 by Fortnow [92], who showed that no random-access machine can solve 3Sat simultaneously in n1+o(1) time and n1−ε space, for any ε > 0. Later, Lipton and Viglas [176] gave a different tradeoff involving a striking exponent; I’ll use their result as my running example in this section. Theorem √ 64 (Lipton-Viglas [176]) No random-access machine can solve 3Sat simultaneously
1387. u1387 2−ε time and no(1) space, for any ε > 0.
1388. u1388 in n
1389. u1389 Here, a “random-access machine” means a machine that can access an arbitrary memory location in O (1) time, as usual in practical programming. This makes Theorem 64 stronger than one might have assumed: it holds not merely for unrealistically weak models such as Turing machines, but for
1390. u1390 Also, “no(1) space” means that we get the n-bit 3Sat instance itself “realistic” models as well.63 in a read-only memory, and we also get no(1) bits of read/write memory to use as we wish.
1391. u1391 61Indeed, the hybrid circuit lower bounds of Section 6.3.2 could already be considered examples of ironic complexity theory. In this section, we discuss other examples.
1392. u1392 62On unrealistic models such as one-tape Turing machines, one can prove up to Ω
1393. u1393 lower bounds for 3Sat and many other problems (even recognizing palindromes), but only by exploiting the fact that the tape head needs to waste a lot of time moving back and forth across the input.
1394. u1394 63Some might argue that Turing machines are more realistic than RAM machines, since Turing machines take into account that signals can propagate only at a finite speed, whereas RAM machines don’t!
1395. u1395 However, RAM machines are closer to what’s assumed in practical algorithm development, whenever memory latency is small enough to be treated as a constant.
1396. u1396 While Theorem 64 is obviously a far cry from P̸ = NP, it does rely essentially on 3Sat being NP-complete: we don’t yet know how to prove analogous results for matching, linear programming, or other natural problems in P.64
1397. u1397 This makes Theorem 64 fundamentally different from (say) the Parity /∈AC0 result of Section 6.2.3.
1398. u1398 Let DTISP (T, S) be the class of languages decidable by an algorithm, running on a RAM machine, that uses O (T) time and O (S) space. Then Theorem 64 can be stated more succinctly as
1399. u1399 for all ε > 0.
1400. u1400 At a high level, Theorem 64 is proved by assuming the opposite, and then deriving stranger and stranger consequences until we ultimately get a contradiction with the Nondeterministic Time Hierarchy Theorem (Theorem 38). There are three main ideas that go into this. The first idea is a tight version of the Cook-Levin Theorem (Theorem 2). In particular, one can show, not merely that 3Sat is NP-complete, but that 3Sat is complete for NTIME (n) (that is, nondeterministic linear-time on a RAM machine) under nearly linear-time reductions—and moreover, that each individual bit of the 3Sat instance is computable quickly, say in logO(1) n time. This means that, to prove Theorem 64, it suffices to prove a non-containment of complexity classes:
1401. u1401 for all ε > 0.
1402. u1402 The second idea is called “trading time for alternations.” Consider a deterministic computation that runs for T steps and uses S bits of memory.
1403. u1403 Then we can “chop the computation up” into k blocks, B1, . . . , Bk, of T/k steps each.
1404. u1404 The statement that the computation accepts is then equivalent to the statement that there exist S-bit strings x0, . . . , xk, such that
1405. u1405 (i) x0 is the computation’s initial state,
1406. u1406 (ii) for all i ∈{1, . . . , k}, the result of starting in state xi−1 and then running for T/k steps is
1407. u1407 (iii) xk is an accepting state.
1408. u1408 where the Σ2 means that we have two alternating quantifiers: √ an existential quantifier over x1, . . . , xk, followed by a universal quantifier over i.
1409. u1409 T/S to optimize the bound then gives us
1410. u1410 64On the other hand, by proving size-depth tradeoffs for so-called branching programs, researchers have been able
1411. u1411 Unlike the 3Sat tradeoffs, the branching program to obtain time-space tradeoffs for certain special problems in P.
1412. u1412 tradeoffs involve only slightly superlinear time bounds; on the other hand, they really do represent a fundamentally different way to prove time-space tradeoffs, one that makes no appeal to NP-completeness, diagonalization, or hierarchy theorems.
1413. u1413 As one example, in 2000 Beame et al. [41], building on earlier work by Ajtai [18], used branching programs to prove the following: there exists a problem in P, based on binary ( quadratic √ forms, for ) which any RAM algorithm (even a nonuniform one) that uses n1−Ω(1) space must also use Ω
1414. u1414 So in particular,
1415. u1415 The third idea is called “trading alternations for time.” If we assume by way of contradiction that
1416. u1416 then in particular, for all b ≥1, we can add an existential quantifier to get
1417. u1417 So putting everything together, ( if we consider a constant c > 1, and use padding (as in Proposi-
1418. u1418 rather than NTIME (n), then the starting assumption that 3Sat tion 17) to talk about NTIME is solvable in nc−ε time and no(1) space implies that
1419. u1419 But if c2 < 2, then this contradicts the Nondeterministic Time Hierarchy Theorem (Theorem 38).
1420. u1420 Notice that the starting hypothesis about 3Sat was This completes the proof of Theorem 64.
1421. u1421 applied not once but twice, which was how the final running time became nc2.
1422. u1422 A proof of this general form, making a sequence of trades between running time and nondeterminism, is called an alternating-trading proof. Later, using a more involved alternating-trading proof, Fortnow and van Melkebeek [95] improved Theorem 64, to show √ that 3Sat can’t be solved by a RAM machine using nϕ−ε time and no(1) space, where ϕ = 1+
1423. u1423 ≈1.618 √ is the golden ratio. Subsequently Williams [271] improved the time bound still further to n
1424. u1424 3−ε, and then [273] to n2 cos π/7−ε.
1425. u1425 In 2012, however, Buss and Williams [69] showed that no alternation-trading proof can possibly improve that exponent beyond the peculiar constant 2 cos π/7 ≈1.801.
1426. u1426 There have been many related time-space tradeoffresults, including for #P-complete and PSPACE-complete problems, but I won’t cover them here (see van Melkebeek [181] for a survey).
1427. u1427 Alternation-trading has had applications in complexity theory other than to time-space tradeoffs.
1428. u1428 In particular, it played a key role in a celebrated 1983 result of Paul, Pippenger, Szemeredi, and Trotter [207], whose statement is tantalizingly similar to P̸ = NP.
1429. u1429 Theorem 65 (Paul et al. [207]) TIME (n)̸ = NTIME (n), if we define these classes using multipletape Turing machines.
1430. u1430 In this case, the key step was to show, via a clever combinatorial argument involving “pebble games,” that for multi-tape Turing machines, deterministic linear time can be simulated in Σ4TIME (f (n)), for some f that’s slightly sublinear. This, combined with the assumption TIME (n) = NTIME (n), is then enough to produce a contradiction with a time hierarchy theorem.
1431. u1431 All the results mentioned above clearly evade the natural proofs barrier, because they ultimately ( √ rely on diagonalization, and (more to the point) because classes like TIME (n) and DTISP
1432. u1432 contain plausible pseudorandom function candidates. Whether they evade the relativization barrier (let alone algebrization) is a trickier question; it depends on subtle details of the oracle access mechanism. There are some definitions of the classes TIME (n)A, DTISP (T, S)A, and so on under which these results relativize, and others under which they don’t: for details, see for example Moran [185].
1433. u1433 On the definitions that cause these results not to relativize, the explanation for how is that the proofs “look inside” the operations of a RAM machine or a multi-tape Turing machine just enough for something to break down if certain kinds of oracle calls are present. To illustrate, in the proof of Theorem 64 above, we nondeterministically guessed the complete state of the machine at various steps in its execution, taking advantage of the fact that the state was an no(1)-bit string.
1434. u1434 This wouldn’t have worked had there been an n-bit query written onto an oracle tape (even if the oracle tape were write-only). Likewise, in the proof of Theorem 65, the combinatorial pebble arguments use specific properties of multi-tape Turing machines that might fail for RAM machines, let alone for oracle machines.
1435. u1435 Because their reasons for failing to relativize have nothing to do with lifting to large finite fields, I conjecture that, with some oracle access mechanisms, Theorems 64 and 65 would also be non-algebrizing. But this remains to be shown.
1436. u1436 6.4.2 In Section 6.2.4, we saw how Smolensky [243] and Razborov [221] proved strong lower bounds against the class AC0 [p], or constant-depth, polynomial-size circuits of AND, OR, NOT, and MOD
1437. u1437 This left the frontier of circuit lower bounds as AC0 [m], where m is p gates, where p is prime.
1438. u1438 composite.
1439. u1439 Meanwhile, we saw in Section 6.3 how Buhrman, Fortnow, and Thierauf [62] proved that MAEXP̸ ⊂P/poly, but how this can’t be extended even to NEXP̸ ⊂P/poly using algebrizing techniques. Indeed, it remains open even to prove NEXP̸ ⊂TC0.
1440. u1440 This state of affairs—and its continuation for decades—helps to explain why many theoretical computer scientists were electrified when Ryan Williams proved the following in 2011. Theorem 66 (Williams [280]) NEXP̸ ⊂ACC (and indeed NTIME (2n)̸ ⊂ACC), where ACC is the union of AC0 [m] over all constants m.65
1441. u1441 If we compare it against the ultimate goal of proving NP̸ ⊂P/poly, Theorem 66 looks laughably weak: it shows only that Nondeterministic Exponential Time, a class vastly larger than NP, is not in ACC, a circuit class vastly smaller than P/poly.
1442. u1442 But a better comparison is against where we were before.
1443. u1443 The proof of Theorem 66 was noteworthy not only because it defeats all the known barriers (relativization, algebrization, and natural proofs), but also because it brings together almost all known techniques in Boolean circuit lower bounds, including diagonalization, the polynomial method, interactive proof results, and ironic complexity theory. So it’s worth at least
1444. u1444 65We can also allow MOD-m1 gates,
1445. u1445 this is equivalent to AC0 [lcm (m1, m2, . . .)].
1446. u1446 On the other hand, if we allow MOD-m gates for non-constant m (and in particular, for m growing polynomially with n), then we jump up to TC0.
1447. u1447 sketching the elaborate proof, so we can see how a lower bound at the current frontier operates. (For further details, I recommend two excellent expository articles by Williams himself [274, 278].)
1448. u1448 At a stratospherically high level, the proof of Theorem 66 is built around the Nondeterministic Time Hierarchy Theorem, following a program that Williams had previously laid out in [276]. More concretely, we assume ( that NTIME (2n) ⊂ACC.
1449. u1449 We then use that assumption to show that NTIME (2n) = NTIME
1450. u1450 for some positive k: a slight speedup of nondeterministic machines, but enough to achieve a contradiction with Theorem 38.
1451. u1451 How do we use the assumption NTIME (2n) ⊂ACC to violate the Nondeterministic Time Hierarchy Theorem? The key to this—and this is where “ironic complexity theory” enters the story—is a faster-than-brute-force algorithm for a problem called ACCSat.
1452. u1452 Here we’re given as input a description of ACC circuit C, and want to decide whether there exists an input x ∈{0, 1}n such that C (x) = 1. The core of Williams’s proof is the following straightforwardly algorithmic result. Lemma 67 (Williams [280]) There’s a deterministic algorithm that solves ACCSat, for ACC circuits of depth d with n inputs, in 2n−Ω(nδ) time, for some constant δ > 0 that depends on d.
1453. u1453 The proof of Lemma 67 is itself a combination of several ideas. First, one appeals to a powerful structural result of Yao [285], Beigel-Tarui [42], and Allender-Gore [24] from the 1990s, which shows that functions in ACC are representable in terms of low-degree polynomials.
1454. u1454 Lemma 68 ([285, 42, 24]) Let f : {0, 1}n →{0, 1} be computable by an AC0 [m] circuit of size
1455. u1455 Then f (x) can be ( expressed ) as g (p (x)), where p : {0, 1}n →N is a polynomial of s and depth d.
1456. u1456 degree logO(1) s that’s a sum of exp
1457. u1457 monomials with coefficients of 1, and g : N →{0, 1}
1458. u1458 ) is some efficiently computable function.
1459. u1459 Moreover, this conversion can be done in exp time.
1460. u1460 (Here the constant in the big-O depends on both d and m.)
1461. u1461 The proof of Lemma 68 uses some elementary number theory, and is closely related to the polynomial method from Section 6.2.4, by which one shows that any AC0 [p] function can be approximated by a low-degree polynomial over the finite field Fp.66
1462. u1462 In proving Lemma 67, next one devises a faster-than-brute-force algorithm that, given a function g (p (x)) as above, decides whether there exists an x ∈{0, 1}n such that g (p (x)) = 1.
1463. u1463 The first step is to give an ( algorithm that constructs a table of all 2n values of p (x), for all the 2n possible
1464. u1464 nO(1) time, rather than the O (2ns) time that one would need na¨ıvely. values of x, in (In other words, this algorithm uses only nO(1) time on average per entry in the table, rather than O (s) time—an improvement if s is superpolynomial.) Here there are several ways to go: one can use a fast rectangular matrix multiplication algorithm due to Coppersmith [78], but one can also just use a dynamic programming algorithm reminiscent of the Fast Fourier Transform.
1465. u1465 Now, by combining this table-constructing algorithm with Lemma 68, we can immediately solve ACCSat, for an ACC circuit of size s = 2nδ, in 2nnO(1) time, which is better than the O (2ns) time that we would need na¨ıvely.
1466. u1466 But this still isn’t good enough to prove Lemma 67, which demands a 2n−Ω(nδ) algorithm.
1467. u1467 So there’s a further trick: given an ACC circuit C of size nO(1), we first “shave off” nδ of the n variables, building a new ACC circuit C′ that takes as input the n −nδ remaining variables, and that computes the OR of C over all 2nδ possible assignments to the nδ
1468. u1468 66Interestingly, both the polynomial method and the proof of Lemma 68 are also closely related to the proof of Toda’s Theorem (Theorem 13), that PH ⊆P#P.
1469. u1469 shaved variables.67 The new circuit C′ has size 2O(nδ), so we can construct the table, and thereby solve ACCSat for C′ (and hence for C), in time 2n−Ω(nδ).
1470. u1470 Given Lemma 67, as well as the starting assumption ( NTIME (2n) ⊂ACC, there’s still a lot of work to do to prove that NTIME (2n) = NTIME
1471. u1471 . Let me summarize the four main steps:
1472. u1472 (1) We first use a careful, quantitative version of the Cook-Levin Theorem (Theorem 2), to reduce
1473. u1473 the problem of simulating an NTIME (2n) machine to a problem called Succinct3Sat.
1474. u1474 In
1475. u1475 that problem, we’re given a circuit C whose truth table encodes an exponentially large 3Sat
1476. u1476 instance φ, and the problem is to decide whether or not φ is satisfiable.
1477. u1477 Indeed, we prove
1478. u1478 something stronger: the circuit C can be taken to be an AC0 circuit.68
1479. u1479 (2) We next appeal to a result of Impagliazzo, Kabanets, and Wigderson [134], which says that if
1480. u1480 NEXP ⊂P/poly, then the satisfying assignments for satisfiable Succinct3Sat instances can
1481. u1481 themselves be constructed by polynomial-size circuits.
1482. u1482 (3) We massage the result (2) to get a conclusion about ACC: roughly speaking, if NEXP ⊂ACC,
1483. u1483 then a satisfying assignment for an AC0 Succinct3Sat instance Φ can itself be constructed
1484. u1484 Furthermore, the problem of verifying that W does indeed encode a
1485. u1485 satisfying assignment for Φ can be solved in slightly less than 2n time nondeterministically,
1486. u1486 if we use the fact (Lemma 67) that ACCSat is solvable in 2n−Ω(nδ) time.
1487. u1487 (4) Putting everything together, we get that NTIME (2n) machines can be reduced to AC0 Suc-
1488. u1488 cinct3Sat instances, which can ( then (assuming NEXP ⊂ACC, and using the ACCSat al-
1489. u1489 But that contradicts the Non-
1490. u1490 deterministic Time Hierarchy Theorem (Theorem 38).
1491. u1491 Let me mention some improvements and variants of Theorem 66. Already in his original paper [280], Williams noted that the proof actually yields a stronger result, that NTIME (2n) has no ACC circuits of “third-exponential” size: that is, size f (n) where f (f (f (n))) grows exponentially. He also gave a second result, that TIME (2n)NP—that is, deterministic exponential time with an NP oracle—has no ACC circuits of size 2no(1).
1492. u1492 More recently, Williams has extended Theorem 66 to show that NTIME (2n) /1 ∩coNTIME (2n) /1 (where the /1 denotes 1 bit of nonuniform advice) doesn’t have ACC circuits of size nlog n [277], and also to show that even ACC circuits of size nlog n with threshold gates at the bottom layer can’t compute all languages in NEXP [279].
1493. u1493 At this point, I should step back and make some general remarks about the proof of Theorem 66 and the prospects for pushing it further. First of all, why did this proof only yield lower bounds for functions in the huge complexity class NEXP, rather than EXP or NP or even P?
1494. u1494 The short answer is that, in order to prove that a class C is not in ACC via this approach, we need to use the assumption C ⊂ACC to violate a hierarchy theorem for C-like classes. But there’s a bootstrapping problem: the mere fact that C has small ACC circuits doesn’t imply that we can find those circuits in a C-like class, in order to obtain the desired contradiction.
1495. u1495 When C = NEXP, we can use the
1496. u1496 67Curiously, this step can only be applied to the ACC circuits themselves, which of course allow OR gates. It can’t be applied to the Boolean functions of low-degree polynomials that one derives from the ACC circuits.
1497. u1497 68In Williams’s original paper [280], this step required invoking the NEXP ⊂ACC assumption (and only yielded an ACC circuit).
1498. u1498 But subsequent improvements have made this step unconditional: see for example, Jahanjou, Miles, and Viola [137].
1499. u1499 nondeterministic guessing power of the NTIME classes simply to guess the small ACC circuits for NEXP, but even when C = EXP this approach seems to break down.
1500. u1500 A second question is: what in Williams’s proof was specific to ACC?
1501. u1501 Here the answer is that the proof used special properties of ACC in one place only: namely, in the improved algorithm for ACCSat (Lemma 67).
1502. u1502 This immediately suggests a possible program to prove NEXP̸ ⊂C for larger and larger circuit classes C.
1503. u1503 For example, let TC0Sat be the problem where we’re given as input a TC0 circuit C (that is, a neural network, or constant-depth circuit of threshold gates), and we want to decide whether there exists an x ∈{0, 1}n such ( that C (x) = 1.
1504. u1504 Then if we could solve TC0Sat even slightly faster than brute force—say, in O
1505. u1505 time for some positive k— Williams’s results would immediately imply NEXP̸ ⊂TC0.69 Likewise, recall from Section ( 2.1 that
1506. u1506 2n/nk) that CircuitSat is the satisfiability problem for arbitrary Boolean circuits.
1507. u1507 algorithm for CircuitSat, then Williams’s results would imply the long-sought NEXP̸ ⊂P/poly.
1508. u1508 Admittedly, one might be skeptical that faster-than-brute-force algorithms should even exist for problems like TC0Sat and CircuitSat. But Williams and others have addressed that particular worry, by showing that circuit lower bounds for NEXP would follow even from faster-than-bruteforce derandomization algorithms: that is, deterministic algorithms to find satisfying assignments under the assumption that a constant fraction of all assignments are satisfying. Obviously there’s a fast randomized algorithm R under that assumption: namely, keep picking random assignments until you find one that works! Thus, to prove a circuit lower bound, “all we’d need to do” is give a nontrivial deterministic simulation of R—something that would necessarily exist under standard derandomization hypotheses (see Section 5.4.1). More precisely:
1509. u1509 Theorem 69 (Williams [276], Santhanam-Williams [229]) Suppose there’s a deterministic
1510. u1510 algorithm, running in 2n/f (n) time for any superpolynomial function f, to decide whether a polynomial-size Boolean circuit C : {0, 1}n →{0, 1} has no satisfying assignments or at least 2n−2 satisfying assignments, promised that one of those is the case.
1511. u1511 Then NEXP̸ ⊂P/poly.70 Likewise, if such an algorithm exists for TC0 circuits, then NEXP̸ ⊂TC0.
1512. u1512 Theorem 69 means that, besides viewing Williams’s program as “ironic complexity theory,” we can also view it as an instance of circuit lower bounds through derandomization, an idea discussed in Section 6.
1513. u1513 In addition to derandomization, one could use faster algorithms for certain classic computational problems to prove circuit lower bounds. These algorithms might or might not exist, but it’s at least plausible that they do. Thus, recall Theorem 24, which said that if CircuitSAT for circuits of depth o (n) requires (2 −o (1))n time, then EditDistance requires nearly quadratic time. In Section 5.1, we interpreted this to mean that there’s a plausible ( hardness conjecture—albeit,
1514. u1514 algorithm for EditDistance one much stronger than P̸ = NP—implying that the classic O
1515. u1515 But there’s a different way to interpret the same connection: namely, if Edis nearly optimal.
1516. u1516 itDistance were solvable in less than ∼n2 time, then various satisfiability problems would be
1517. u1517 69Very recently, Kane and ( Williams ) [142] managed to give an explicit Boolean function that requires depth-2 threshold circuits with Ω
1518. u1518 However, their argument doesn’t proceed via a better-than-bruteforce algorithm for depth-2 TC0Sat.
1519. u1519 Even more recently, Chen, Santhanam, and Srinivasan [71] gave the first nontrivial algorithm for TC0Sat with a slightly-superlinear number of wires.
1520. u1520 This wasn’t enough for a new lower bound, but Chen et al. also used related ideas to show that Parity can’t be computed even by polynomial-size AC0 circuits with no(1) threshold gates.
1521. u1521 70This strengthens a previous result of Impagliazzo, Kabanets, and Wigderson [134], who showed that, if such a deterministic algorithm exists that runs in 2no(1) time, then NEXP̸ ⊂P/poly.
1522. u1522 solvable in less than ∼2n time, and we’ve already seen that the latter would lead to new circuit lower bounds!
1523. u1523 Putting all this together, Abboud et al. [11] recently proved the following striking theorem, which they describe in slogan form as “a polylog shaved is a lower bound made”:
1524. u1524 )
1525. u1525 n2/ logc n Theorem 70 (Abboud et al. [11]) Suppose that EditDistance is solvable in O time, for every constant c.
1526. u1526 A third question is: how does the proof of Theorem 66 evade the known barriers?
1527. u1527 Because of the way the algorithm for ACCSat exploits the structure of ACC circuits, we shouldn’t be surprised if the proof evades the relativization and algebrization barriers.
1528. u1528 And indeed, using the techniques of Wilson [282] and of Aaronson and Wigderson [10], one can easily construct an oracle A such that NEXPA ⊂ACCA, and even an algebraic oracle eA such that NEXP
1529. u1529 A, e thereby showing that NEXP̸ ⊂ACC is non-relativizing and non-algebrizing.
1530. u1530 Meanwhile, because it uses diagonalization (in the form of the Nondeterministic Time Hierarchy Theorem), we might say that the proof of Theorem 66 has the “capacity” to evade natural proofs. On the other hand, as I alluded to in Section 6.2.5, it’s not yet clear whether ACC is powerful enough to compute pseudorandom functions—and thus, whether it even has a natural proofs barrier to evade! The most we can say is that if ACC has a natural proofs barrier, then Theorem 66 evades it.
1531. u1531 Given everything we saw in the previous sections, a final question arises: is there some fourth barrier, beyond relativization, algebrization, and natural proofs, which will inherently prevent even Williams’s techniques from proving P̸ = NP, or even (say) NEXP̸ ⊂TC0? One reasonable answer is that this question is premature: in order to identify the barriers to a given set of techniques, we first need to know formally what the techniques are—i.e., what properties all the theorems using those techniques have in common—but we can’t know that until the techniques have had a decade or more to solidify, and there are at least three or four successful examples of their use. Of course, one obvious “barrier” is that, while Theorem 69 shows that we could get much further just from plausible derandomization assumptions, eventually we might find ourselves asking for faster-thanbrute-force algorithms that simply don’t exist—in which case, ironic complexity theory would’ve run out of the irony that it needs as fuel.
1532. u1532 Stepping back, I see Theorem 66 as having contributed something important to the quest to prove P̸ = NP, by demonstrating just how much nontrivial work can get done, and how many barriers can be overcome, along the way to applying a 1960s-style hierarchy theorem.
1533. u1533 Williams’s result makes it possible to imagine that, in the far future, P̸ = NP might be proved by assuming the opposite, then deriving stranger and stranger consequences using thousands of pages of mathematics barely comprehensible to anyone alive today—and yet still, the coup de grˆace will be a diagonalization, barely different from what Turing did in 1936.
1534. u1534 6.5
1535. u1535 Arithmetic Complexity Theory Besides Turing machines and Boolean circuits acting on bits, there’s another kind of computation that has enormous relevance to the attempt to prove P̸ = NP. Namely, we can consider computer programs that operate directly on elements of a field, such as the reals or complex numbers. Perhaps the easiest way to do this is via arithmetic circuits, which take as input a collection of elements x1, . . . , xn of a field F,71 and whose operations consist of adding or multiplying any two previous elements—or any previous element and any scalar from F—to produce a new F-element. We then
1536. u1536 71I’ll restrict to fields here for simplicity, but one can also consider (e.g.) rings.
1537. u1537 consider the minimum number of operations needed to compute some polynomial g : Fn →F, as a function of n.
1538. u1538 For concreteness, we can think of F as the reals R, although we’re most interested in algorithms that work over any F.
1539. u1539 Note that, if we work over a finite field Fm, then we need to specify whether we want to compute g as a formal polynomial, or merely as a function over Fm.72
1540. u1540 At first glance, arithmetic circuits seem more powerful than Boolean circuits, because they have no limit of finite precision: for example, an arithmetic circuit could multiply π and e in a single time step.
1541. u1541 From another angle, however, arithmetic circuits are weaker, because they have no facility (for example) to extract individual bits from the binary representations of the F elements: they can only manipulate them as F elements. In general, the most we can say is that, if an input has helpfully been encoded using the elements 0, 1 ∈F only, then an arithmetic circuit can simulate a Boolean one, by using x →1−x to simulate NOT, multiplication to simulate Boolean AND, and so on. But for arbitrary inputs, such a simulation might be impossible.
1542. u1542 Thus, arithmetic circuits represent a different kind of computation: or rather, a generalization of the usual kind, since we can recover ordinary Boolean computation by setting F = F2. A major reason to focus on arithmetic circuits is that it often seems easier—or better, less absurdly hard!— to understand circuit size in the arithmetic setting than in the Boolean one. The usual explanation given for this is the so-called “yellow books argument”: arithmetic complexity brings us closer to continuous mathematics, about which we have centuries’ worth of deep knowledge (e.g., algebraic geometry and representation theory) that’s harder to apply in the Boolean case.
1543. u1543 One remark: in the rest of the section, I’ll talk exclusively about arithmetic circuit complexity: that is, about nonuniform arithmetic computations, and the arithmetic analogues of questions such as NP versus P/poly (see Section 5.2).
1544. u1544 But it’s also possible to develop a theory of arithmetic Turing machines, which (roughly speaking) are like arithmetic circuits except that they’re uniform, and therefore need loops, conditionals, memory registers, and so on. See the book of Blum, Cucker, Shub, and Smale (BCSS) [51] for a beautiful exposition of this theory. In the BCSS framework, one can ask analogues of the P ?= NP question for Turing machines over arbitrary fields F, such as R or C, recovering the “ordinary, Boolean” P ?= NP question precisely when F is finite.
1545. u1545 At present, no implications are known among the P ?= NP, the PR
1546. u1546 ?= NPC questions, although it’s known that PC = NPC implies NP ⊂P/poly (see for example B¨urgisser [63, Chapter 8]).73
1547. u1547 The problems of proving PR̸ = NPR and PC̸ = NPC are known to be closely related to the problem of proving arithmetic circuit lower bounds, which we’ll discuss in the following sections. I can’t resist giving one example of a connection, due to BCSS [51].
1548. u1548 Given a positive integer n, let τ (n) be the number of operations in the smallest arithmetic circuit that takes the constant 1 as its sole input, and that computes n using additions, subtractions, and multiplications.
1549. u1549 For example, we have
1550. u1550 Also, let τ ∗(n) be the minimum of τ (kn) over all positive integers k.
1551. u1551 72To illustrate, 0 and x2 + x are equal as functions over the finite field F2, but not equal as formal polynomials.
1552. u1552 ?= NPC questions is simply that, because R is an ordered
1553. u1553 73The central difference between the PR field, one defines Turing machines over R to allow comparisons (<, ≤) and branching on their results.
1554. u1554 Theorem 71 (BCSS [51]) Suppose τ ∗(n!) grows faster than (log n)O(1).
1555. u1555 6.5.1
1556. u1556 Permanent Versus Determinant
1557. u1557 ?= NP is the “flagship problem” of Boolean complexity theory, so the central, flagship Just as P problem of arithmetic complexity is that of permanent versus determinant. This problem concerns the following two functions of an n × n matrix X ∈Fn×n:
1558. u1558 Despite the similarity of their definitions—they’re identical apart from the (−1)sgn(σ)—the permanent and determinant have dramatic differences.
1559. u1559 The determinant is computable in polynomial time, for example by using Gaussian elimination. (Indeed, the determinant is computable in O (nω) time, where ω ∈[2, 2.373] is the matrix multiplication exponent; see Section 4.) The determinant has many other interpretations—for example, the product of X’s eigenvalues, and the volume of the parallelepiped spanned by its row vectors—giving it a central role in linear algebra and geometry.
1560. u1560 By contrast, Valiant [262] proved in 1979 that the permanent is #P-complete.
1561. u1561 Thus, a polynomial-time algorithm for the permanent would imply even more than P = NP: it would yield an efficient algorithm not merely to solve NP-complete problems, but to count how many solutions they have.
1562. u1562 In some sense, the #P-completeness of the permanent helps to explain why Per, unlike Det, has no simple geometric or linear-algebraic interpretations: if such interpretations existed, then they might imply P = P#P.
1563. u1563 In the arithmetic model, Kaltofen and Villard [141] constructed circuits of size O
1564. u1564 that compute Det (X) as a formal polynomial in the entries of X, and that work over an arbitrary field F.75
1565. u1565 By contrast, Valiant conjectured the following.
1566. u1566 Conjecture 72 (Valiant’s Conjecture) Any arithmetic circuit for Per (X) requires size superpolynomial in n, over any field of characteristic other than 2.76,77
1567. u1567 74In later work, B¨urgisser [65] showed that the same conjecture about τ ∗(n!) (called the τ-conjecture) would also imply Valiant’s Conjecture 72, that ( the permanent has no polynomial-size arithmetic circuits.
1568. u1568 75One might think that size-O
1569. u1569 circuits for the determinant would be trivial, because of Gaussian elimination, and that one could even get O (nω), where ω ≤2.373 is the matrix multiplication exponent. However, the difficulty is that Gaussian elimination involves division, whereas the model as we’ve defined it allows only addition and multiplication.
1570. u1570 A seminal result of Strassen [251] says that division gates can always eliminated in an arithmetic circuit to compute a polynomial, leaving addition and multiplication gates only—but doing so could produce a polynomial blowup in the circuit ( size.
1571. u1571 In the case of Gaussian elimination, a na¨ıve application of Strassen’s method yields a circuit of size O
1572. u1572 76Fields of characteristic 2, such as F2, are a special case: there, the permanent and determinant are equivalent, so in particular Per (X) has polynomial-size arithmetic circuits.
1573. u1573 77In the literature, Conjecture 72 is often called the VP̸ = VNP conjecture, with VP and VNP being arithmetic analogues of P and NP respectively. I won’t use that terminology in this survey, for several reasons: (1) VP is arguably more analogous to NC than to P, (2) VNP is arguably more analogous to #P than to NP, and (3) Conjecture 72 is almost always studied as a nonuniform conjecture, more analogous to NP̸ ⊂P/poly than to P̸ = NP.
1574. u1574 B¨urgisser [64] showed that, if Conjecture 72 fails over any field of positive characteristic, or if it fails over any field of characteristic zero and the Generalized Riemann Hypothesis holds, then P#P ⊂P/poly, and hence NP ⊂P/poly.78
1575. u1575 (The main difficulty in proving this result is just that an arithmetic circuit might have very large constants hardwired into it.)
1576. u1576 On the other hand, no converses to this result are currently known.
1577. u1577 It’s conceivable, for example, that we could have P = P#P for some “inherently Boolean” reason, even if the permanent required arithmetic circuits of exponential size. To put it another way, Conjecture 72 could serve as an “arithmetic warmup”— some would even say an “arithmetic prerequisite”—to Boolean separations such as P#P̸ ⊂P/poly and P̸ = NP.
1578. u1578 Better yet, Conjecture 72 turns out to be implied by (and nearly equivalent to) an appealing mathematical conjecture, which makes no direct reference to computation or circuits.
1579. u1579 Let’s say that the n × n permanent linearly embeds into the m × m determinant, if it’s possible to express Per (X) (for an n × n matrix X ∈Fn×n) as Det (L (X)), where L (X) is an m × m matrix each of whose entries is an affine combination of the entries of X. Then let D (n) be the smallest m such that the n × n permanent linearly embeds into the m × m determinant.
1580. u1580 Theorem 73 (Grenet [111]) D (n) ≤2n −1.
1581. u1581 (in this case, as we see, Theorem 73 is not tight), while when n = 3 we have
1582. u1582 By contrast, the best current lower bound on D (n) is quadratic, and was proved by Mignon and Ressayre [182] in 2004, following a long sequence of linear lower bounds: Theorem 74 (Mignon and Ressayre [182]) D (n) ≥n2/2.
1583. u1583 (Actually, Mignon and Ressayre proved Theorem 74 only for fields of characteristic 0.
1584. u1584 Their result was then extended to all fields of characteristic other than 2 by Cai, Chen, and Li [70] in 2008.)
1585. u1585 The basic idea of the proof of Theorem 74 is to consider the Hessian matrix of a polynomial p : FN →F, or the matrix of second partial derivatives, evaluated at some particular point X0 ∈FN:
1586. u1586 78Indeed, #P would even have polynomial-size circuits of depth logO(1) n.
1587. u1587 Here we mean the “formal” partial derivatives of p: even if F is a finite field, we can still symbolically differentiate a polynomial over F, to produce new polynomials over smaller sets of variables. In general, when we’re trying to lower-bound the difficulty of computing a polynomial p, a common technique in arithmetic complexity is to look at various partial derivatives
1588. u1588 ∂xi1···∂xik ∂kp —and in particular, at the dimensions of vector spaces spanned by those partial derivatives, or the ranks of matrices formed from them—and then argue that, if p had a small circuit (or formula, or whatever), then those dimensions or ranks couldn’t possibly be as high as they are.
1589. u1589 In the case of Theorem 74, we prove the following two statements:
1590. u1590 (1) If p is the permanent, of an n × n matrix of N = n2 indeterminates, then there exists a point
1591. u1591 X0 ∈FN such that rank (Hp (X0)) = N.
1592. u1592 (2) If p is the determinant of an m × m matrix of affine functions in the N indeterminates, then
1593. u1593 Combining these, we get m ≥n2/2, if p is both the n×n permanent and an m×m determinant.
1594. u1594 So to summarize, the “blowup” D (n) in embedding the permanent into the determinant is known to be at least quadratic and at most exponential.
1595. u1595 The huge gap here becomes a bit less surprising, once we know that D (n) is tightly connected to the arithmetic circuit complexity of the permanent.
1596. u1596 In particular, recall that a formula is just a circuit in which every gate has a fanout of 1. Then Valiant [261] showed the following: Theorem 75 (Valiant [261]) D (n) ≤F (n)+1, where F (n) is the size of the smallest arithmetic formula for the n × n permanent.
1597. u1597 Thus, if we could prove that D (n) grew faster than any polynomial, we’d have shown that the permanent has no polynomial-size formulas. But heightening the interest still further, Valiant et al. [264] showed that in the arithmetic world, there’s a surprisingly tight connection between formulas and circuits: Theorem 76 (Valiant et al. [264]) If a degree-d polynomial has an arithmetic circuit of size s, then it also has an arithmetic formula of size (sd)O(log d).
1598. u1598 Theorem 76 implies that D (n) ≤C (n)O(log n), where C (n) is the size of the smallest arithmetic circuit for the n × n permanent. This means that, if we could prove that D (n) grew not only superpolynomially but faster than nO(log n), we’d also have shown that C (n) grew superpolynomially, thereby establishing Valiant’s Conjecture 72.
1599. u1599 But lower-bounding D (n) is not merely sufficient for proving Valiant’s Conjecture; ( it’s also necessary! For recall that the n × n determinant has an arithmetic circuit of size O
1600. u1600 [141]. So we get the following chain of implications:
1601. u1601 D (n) > nO(log n) =⇒F (n) > nO(log n) (by Theorem 75)
1602. u1602 =⇒C (n) > nO(1) (by Theorem 76; this is Valiant’s Conjecture 72)
1603. u1603 =⇒D (n) > nO(1) (by the nO(1) arithmetic circuits for determinant)
1604. u1604 Today, a large fraction of the research aimed at proving P̸ = NP is aimed, more immediately, at proving Valiant’s Conjecture 72 (see Agrawal [14] for a survey focusing on that goal). The hope is that, on the one hand, powerful tools from algebraic geometry and other fields can be brought to bear on Valiant’s problem, but on the other, that solving it could provide insight about the original P ?= NP problem.
1605. u1605 6.5.2
1606. u1606 Arithmetic Circuit Lower Bounds I won’t do justice in this survey to the now-impressive body of work motivated by Conjecture 72; in particular, I’ll say little about proof techniques.
1607. u1607 Readers who want to learn more about arithmetic circuit lower bounds should consult Shpilka and Yehudayoff[239, Chapter 3] for an excellent survey circa 2010, or Saraf [230] for a 2014 update.
1608. u1608 Briefly, though, computer scientists have tried to approach Conjecture 72 much as they’ve approached NP̸ ⊂P/poly, by proving lower bounds against more and more powerful arithmetic circuit classes. In that quest, they’ve had some notable successes (paralleling the Boolean successes), but have also run up against some differences from the Boolean case.
1609. u1609 For starters, just as Razborov [219] and others considered monotone Boolean circuits, one can also consider monotone arithmetic circuits (over fields such as R or Q), in which all coefficients need to be positive. Since the determinant involves −1 coefficients, it doesn’t make sense to ask about monotone circuits for Det (X), but one can certainly ask about the monotone circuit complexity of Per (X). And already in 1982, Jerrum and Snir [138] proved the following arithmetic counterpart of Razborov’s Theorem 47: Theorem 77 (Jerrum and Snir [138]) Any monotone circuit for Per (X) requires size 2Ω(n).
1610. u1610 As another example, just as computer scientists considered constant-depth Boolean circuits (the classes AC0, ACC, TC0, and so on), so we can also consider constant-depth arithmetic circuits, which are conventionally denoted ΣΠ, ΣΠΣ, etc. to indicate whether they represent a multivariate polynomial as a sum of products, a sum of product of sums, etc.
1611. u1611 It’s trivial to prove exponential lower bounds on the sizes of depth-two (ΣΠ) circuits: that just amounts to lower-bounding the number of monomials in a polynomial. More interesting is the following result: Theorem 78 (Grigoriev and Karpinski [112], Grigoriev and Razborov [113]) Over a finite field, any ΣΠΣ circuit for Det (X) requires size 2Ω(n).
1612. u1612 (Indeed, this is true even for circuits representing Det (X) as a function.)
1613. u1613 Curiously, over infinite fields, the best lower bound that we have for the determinant is still a much weaker one, due to Shpilka and Wigderson [238]: Theorem 79 (Shpilka ( ) and Wigderson [238]) Over infinite fields, any ΣΠΣ circuit for Det (X) requires size Ω
1614. u1614 79As this survey was being written, Kayal, Saha, and Tavenas [150] announced a ( proof that ) a certain explicit polynomial, albeit not the permanent or determinant, requires ΣΠΣ circuits of size Ω
1615. u1615 , ( over any ) field F. By comparison, as a function of the number of input variables (n2), Shpilka and Wigderson’s Ω
1616. u1616 lower bound for the determinant [238] is “only” quadratic.
1617. u1617 Theorems 78 and 79 are stated for the determinant, although they have analogues for the permanent.
1618. u1618 In any case, these results certainly don’t succeed in showing that the permanent is harder than the determinant.
1619. u1619 The situation is better when we restrict the fanin of the multiplication gates. In particular, by a ΣΠ[a]ΣΠ[b] circuit, let’s mean a depth-4 circuit where every inner multiplication gate has fanin at most a, and every bottom multiplication gate has fanin at most b.
1620. u1620 Then in 2013, Gupta et al. [119] proved the following.
1621. u1621 √n] circuit
1622. u1622 Theorem 80 (Gupta, Kamath, Kayal, and Saptharishi [119]) Any ΣΠ[O(
1623. u1623 for Per (X) or Det (X) requires size 2Ω(
1624. u1624 √n) for such
1625. u1625 Subsequently, Kayal, Saha, and Saptharishi [149] proved a size lower bound of nΩ( circuits, though not for the permanent or determinant but for a different explicit polynomial.
1626. u1626 The situation is also better when we restrict to homogeneous arithmetic circuits.
1627. u1627 These are circuits where every gate is required to compute a homogeneous polynomial: that is, one where all the monomials have the same degree.
1628. u1628 Here Nisan and Wigderson [204] established the following in 1997.
1629. u1629 Theorem 81 (Nisan and Wigderson [204]) Over any field, any homogeneous ΣΠΣ circuit for Det (X) requires size 2Ω(n).
1630. u1630 Going further, in 2014 Kayal, Limaye, Saha, and Srinivasan [148] gave an explicit polynomial in
1631. u1631 √n) (compared to Theorem n variables for which any homogeneous ΣΠΣΠ circuit requires size nΩ(
1632. u1632 √n), as a function of the number of input variables).
1633. u1633 81’s 2Ω(
1634. u1634 It’s natural to wonder: why are we stuck talking about depth-3 and depth-4 arithmetic circuits? Why couldn’t we show that the permanent and determinant have no constant-depth arithmetic circuits of subexponential size? After all, wasn’t arithmetic complexity supposed to be easier than Boolean complexity?
1635. u1635 In 2008, Agrawal and Vinay [16] gave a striking answer to these questions; they called their answer “the chasm at depth four.” In particular, building on the earlier work of Valiant et al. [264] (Theorem 76), Agrawal and Vinay showed that, if we managed to prove strong enough lower bounds for depth-4 arithmetic circuits, then we’d also get superpolynomial lower bounds for arbitrary arithmetic circuits! Here’s one special case of their result: Theorem 82 (Agrawal and Vinay [16]) Suppose that Per (X) requires depth-4 arithmetic circuits (even homogeneous ones) of size 2Ω(n).
1636. u1636 Then Per (X) requires arithmetic circuits of superpolynomial size, and Valiant’s Conjecture 72 holds.
1637. u1637 In some sense, Theorem 82 does for arithmetic circuit complexity what Theorem 52 did for
1638. u1638 Recall that Theorem 52 constructed AC0 circuits of size 2nε for any Boolean circuit complexity.
1639. u1639 language in NLOGSPACE and any constant ε > 0, thereby showing that strong enough size lower bounds for AC0 circuits would entail a breakthrough in separating complexity classes.
1640. u1640 Arguably, though, Theorem 82 is even more surprising, because it reaches a similar conclusion while talking only about a fixed depth (in this case, depth 4), rather than about arbitrary constant depths.80
1641. u1641 80We can also talk about fixed constant depths in the AC0 case, but if we do, the conclusions are weaker.
1642. u1642 For example, if we managed to prove a 2Ω(n) size lower bound against AC0 circuits of depth 3, ( for some language ) L, then we could deduce from Theorem 52 that any NLOGSPACE machine for L would require Ω
1643. u1643 Subsequently, Koiran [158] and Tavenas [257] showed that Valiant’s Conjecture would follow, not merely from a 2Ω(n) size lower bound for homogeneous depth-4 circuits computing the permanent,
1644. u1644 √n). In an even more exciting development, Gupta but from any size lower bound better than nΩ( et al. [118] reduced the depth from four to three (though only for fields of characteristic 0, and no longer allowing homogeneity):
1645. u1645 Theorem 83 (Gupta, Kamath, Kayal, and Saptharishi [118]) Suppose that Per (X) requires
1646. u1646 √n), over fields of characteristic 0. Then Per (X) depth-3 arithmetic circuits of size more than nΩ( requires arithmetic circuits of superpolynomial size, and Valiant’s Conjecture 72 holds.
1647. u1647 These results can be considered extreme versions of the depth reduction of Spira [245] (see Proposition 27). I should mention that all of these results hold, not just for the permanent, but for any homogeneous polynomial of degree nO(1). In particular, by applying their depth reduction “in the opposite direction” for the determinant, Gupta et al. [118] were able to show that there exist depth-3 arithmetic circuits of size nO(
1648. u1648 This provides an interesting counterpoint to the result of Nisan and Wigderson [204] (Theorem 81), which showed that 2Ω(n) gates are needed for the determinant if we restrict to depth-3 homogeneous circuits.
1649. u1649 There are yet other results in this vein, which give yet other tradeoffs. But perhaps we should step back from the flurry of theorems and try to summarize. After decades of research in arithmetic
1650. u1650 √n) on the sizes of depth-3 and depthcircuit complexity, we now have lower bounds of the form nΩ( 4 arithmetic circuits computing explicit polynomials (subject to various technical restrictions). On the other hand, we also have a deep explanation for why the progress has stopped at the specific
1651. u1651 √n): because any lower bound even slightly better than that would already prove Valiant’s bound nΩ( Conjecture, that the permanent is superpolynomially harder than the determinant!
1652. u1652 It’s as if, in arithmetic complexity, we reach a terrifying precipice—beyond which we can no longer walk but need to fly—sooner than we do in the Boolean case.
1653. u1653 And around 2014, we learned exactly where that precipice is and walked right up to it, but we still haven’t jumped.81,82
1654. u1654 In this connection, it’s worth pointing out that, with the exception of Theorem 74 by Mignon and Ressayre [182], none of the results in this section actually differentiate the permanent from the determinant: that is, none of them prove a lower bound for Per (X) better than the analogous lower bound known for Det (X). Eventually, of course, any proof of Valiant’s Conjecture will need to explain why the permanent is harder than the determinant, which is one of the main motivations for the Mulmuley-Sohoni program (see Section 6.6).
1655. u1655 Let me end this section by discussing two striking results of Ran Raz, and one of Pascal Koiran, that didn’t quite fit into the narrative above.
1656. u1656 The first result is a superpolynomial lower bound on the sizes of multilinear formulas. An arithmetic formula is called multilinear if the polynomial computed by each gate is a multilinear polynomial (that is, no variable is raised to a higher power than 1). Notice that the permanent and determinant are both multilinear polynomials. For that reason, they can be computed by multilinear formulas, and it makes sense to ask about the size of the smallest such formulas.
1657. u1657 In a 2004 breakthrough, Raz [214] proved the following.
1658. u1658 81Or perhaps, we’ve jumped many times, but each time hit the bottom rather than flew!
1659. u1659 82As this survey was being written, Forbes, Kumar, and Saptharishi [89] gave yet another interesting result sharpening the contours of the “chasm at depth four”: namely, they showed that lower bounds on homogeneous depth-4 arithmetic circuits to compute Boolean functions (rather than formal polynomials), and which are only “slightly” stronger than lower bounds that have already been shown, would imply a separation between #P and ACC.
1660. u1660 Theorem 84 (Raz [214]) Any multilinear formula for Per (X) or Det (X) requires size nΩ(log n).83
1661. u1661 What made Theorem 84 striking was that there was no restriction on the formula’s depth. The proof was via the random restriction method from Section 6.2.3, combined with the idea (common in arithmetic complexity) of using matrix rank as a progress measure. In more detail, let p : {0, 1}n →R be a polynomial computed by a small multilinear formula: for simplicity, we’ll take p’s inputs to be Boolean. Then basically, we randomly partition p’s input variables into two small sets X = {x1, . . . , xk} and Y = {y1, . . . , yk}, and a large set Z of size n −2k.
1662. u1662 (Here we should imagine, say, k = n1/3.)
1663. u1663 We then randomly fix the variables in Z to 0’s or 1’s, while leaving the
1664. u1664 Next, we define a matrix M ∈R2k×2k, whose rows are indexed by variables in X and Y unfixed.
1665. u1665 the 2k possible assignments to X, whose columns are indexed by the 2k possible assignments to Y , and whose (X, Y ) entry equals p (X, Y, Z). Finally, we prove the following two statements:
1666. u1666 With high probability, M has rank much smaller than 2k. This is the hard part of the proof:
1667. u1667 we use the assumption that p has a small multilinear formula, and then argue by induction
1668. u1668 If p represents the function f of interest to us (say, the permanent or determinant), then
1669. u1669 Together, these yield the desired contradiction, showing that f can’t have had a small multilinear formula after all.
1670. u1670 It seems likely that the lower bound in Theorem 84 could be improved from nΩ(log n) all the way up to 2Ω(n), but this remains open. Raz and Yehudayoff[217] did manage to prove an exponential lower bound for constant-depth multilinear formulas computing the permanent or determinant; and in a separate work [218], they also proved a 2Ω(n) lower bound for “non-cancelling” multilinear formulas computing an explicit polynomial f (not the permanent or determinant).
1671. u1671 Here “noncancelling”—a notion that I defined in [2]—basically means that nowhere in the formula are we allowed to add two polynomials that “almost perfectly” cancel each other out, leaving only a tiny residue.
1672. u1672 Of course, just like with the arithmetic circuit lower bounds discussed earlier, so far all the known multilinear formula lower bounds fail to distinguish the permanent from the determinant.
1673. u1673 The second result of Raz’s concerns so-called elusive functions.
1674. u1674 Given a polynomial curve f : C →Cn, Raz calls f elusive if f is not contained in the image of any polynomial mapping g : Cn−1 →Cn of degree 2. He then proves the following beautiful theorem.
1675. u1675 Theorem 85 (Raz [215]) Suppose there exists an elusive function whose coefficients can be computed in polynomial time.
1676. u1676 Then Per (X) requires arithmetic circuits of superpolynomial size, and Valiant’s Conjecture 72 holds.
1677. u1677 Arguably, this makes Valiant’s Conjecture look even more like a question of pure algebraic geometry than it did before!
1678. u1678 As evidence that the “elusive function” approach to circuit lower bounds is viable, Raz then constructs an explicit f that’s elusive in a weak sense, which is already enough to imply the following new lower bound:
1679. u1679 83An immediate corollary is that any multilinear circuit for Per (X) or Det (X) requires depth Ω
1680. u1680 Theorem 86 (Raz [215]) For every r, there is an explicit polynomial p with n variables and degree O (r), such that any depth-r arithmetic circuit for p (over any field) requires size n1+Ω(1/r).
1681. u1681 Theorem 87 (Koiran [157]) Suppose that any univariate real polynomial, of the form
1682. u1682 has at most (ℓmn)O(1) real zeroes.
1683. u1683 Then Per (X) requires arithmetic formulas of superpolynomial size (and indeed, D (n) > nO(1)).
1684. u1684 In fact it would suffice to upper-bound the number of integer zeroes of such a polynomial by (ℓmn)O(1).
1685. u1685 Note that, if we had asked about complex zeroes, then Theorem 87 would badly fail, because of counterexamples such as x2n −1. But with real zeroes, no counterexample is known, and Theorem 87 once again raises the tantalizing possibility that tools from analysis could be brought to bear on the permanent versus determinant problem.
1686. u1686 6.5.3
1687. u1687 Arithmetic Natural Proofs?
1688. u1688 In Section 6.5.2, we saw arithmetic circuit lower bounds that, again and again, seem to go “right up to the brink” of proving Valiant’s Conjecture, but then stop short.
1689. u1689 Given this, it’s natural to wonder what the barriers are to further progress in arithmetic complexity, and how they relate to the barriers in the Boolean case.
1690. u1690 We’ve already discussed one obvious barrier, which is that eventually we need techniques that work for the permanent but fail for the determinant.
1691. u1691 It might also be interesting to define an arithmetic analogue of the relativization barrier (Section 6.1.2).
1692. u1692 To my knowledge, this hasn’t been done, but my guess is that in the arithmetic setting, the natural choices for oracles would look a lot like the algebraic oracles studied by Aaronson and Wigderson [10] (see Section 6.3.3). With a notion of “oracle” in hand, one could probably show that most arithmetic circuit lower bounds require arithmetically non-relativizing techniques. On the other hand, this wouldn’t be much of an obstruction, since even the results discussed in Section 6.5.2 should already evade the relativization barrier, for the same reason as those of Sections 6.2.3 and 6.2.4.
1693. u1693 In the rest of this section, I’d like to discuss the contentious question of whether or not arithmetic circuit complexity faces a natural proofs barrier, in the sense of Razborov and Rudich [223]. Recall from Section 6.2.5 that a circuit lower bound proof is called natural if, besides proving that the specific function f of interest to us is not in a circuit class C, the proof also provides a polynomialtime algorithm A that takes as input a function’s truth table, and that certifies a 1/nO(1) fraction of all functions as not belonging to C.
1694. u1694 Such an A can be used to distinguish functions in C from random functions with non-negligible bias. Meanwhile, the class C has a natural proofs barrier if C contains pseudorandom function families, which can’t be so distinguished from random functions, and whose existence is therefore incompatible with the existence of A.
1695. u1695 In the arithmetic setting, presumably we’d call a proof natural if it yields a polynomial-time algorithm84 that takes as input, say, the complete output table of a homogeneous degree-d polynomial p : Fn →F over a finite field F, and that certifies a 1/nO(1) fraction of all such polynomials
1696. u1696 84For simplicity, here I’ll assume that we mean an “ordinary” (Boolean) polynomial-time algorithm, though one could also require polynomial-time algorithms in the arithmetic model.
1697. u1697 as not belonging to the arithmetic circuit class C.
1698. u1698 Also, we’d say that C has a natural proofs barrier if C contains pseudorandom polynomial families.
1699. u1699 By this, we mean families of homogeneous degree-d polynomials, ps : Fn →F, that no |F|O(n)-time algorithm can distinguish from uniformly-random homogeneous degree-d polynomials with non-negligible bias. (We can no longer talk about uniformly-random functions, since an algorithm can easily ascertain, for example, that ps is a degree-d polynomial.)
1700. u1700 By exactly the same logic as in the Boolean case, if C is powerful enough to compute pseudorandom polynomials, then no natural proof can show that a polynomial isn’t in C.
1701. u1701 Now, one point that’s not disputed is that all the arithmetic circuit lower bounds discussed in Section 6.5.2 are natural in the above sense.
1702. u1702 I didn’t say much about how the lower bounds are proved, but as mentioned in Section 6.5.1, arithmetic circuit lower bounds generally proceed by finding some parameter α (p) associated with a polynomial p—say, the rank of its Hessian matrix, or the dimension of a vector space spanned by p’s partial derivatives—to use as a “progress measure.” The proof then argues that
1703. u1703 (1) α (p) is large for the specific polynomial p of interest to us (say, the permanent or determinant),
1704. u1704 (2) every gate added to our circuit or formula can only increase α (p) by so much, thereby implying that p requires many gates. Furthermore, virtually any progress measure α that’s a plausible choice for such an argument—and certainly the ones used in the existing results—will be computable in |F|O(n) time, and will be maximized by a random polynomial p of the appropriate degree. Alas, this implies that the argument is natural! If the circuit class C has a natural proofs barrier, then no such argument can possibly prove p /∈C.
1705. u1705 The part that’s controversial is whether arithmetic circuit classes do have a natural proofs barrier.
1706. u1706 To show that they did, we’d need plausible candidates for pseudorandom polynomials— say, homogeneous degree-d polynomials p : Fn →F that actually have small arithmetic circuits, but that look to any efficient test just like random homogeneous polynomials of degree d.
1707. u1707 The trouble is that, while cryptographers know a great deal about how to construct pseudorandom functions, the accepted constructions are all “inherently Boolean”; they don’t work in the setting of low-degree polynomials over a finite field.
1708. u1708 Thus, to take one example, the work of Goldreich, Goldwasser, and Micali (GGM) [107], combined with that of H˚astad et al. [126], shows how to build a pseudorandom function family starting from any one-way function (see Section 5.3.1).
1709. u1709 And indeed, Razborov and Rudich [223] used a variant of the GGM construction in their original paper on natural proofs.
1710. u1710 However, if we try to implement the GGM construction using arithmetic circuits—say, using multiplication for the AND gates, 1 −x for the NOT gates, etc.—we’ll find ( that we’ve produced an arithmetic circuit of nO(1) depth, which computes a polynomial of exp
1711. u1711 As I mentioned in Section 6.2.5, if we’re willing to assume the hardness of specific cryptographic problems, then there are also much more direct constructions of pseudorandom functions, which produce circuits of much lower depth. In particular, there’s the construction of Naor and Reingold [200], which is based on factoring and discrete logarithm; and that of Banerjee et al. [39], which is based on noisy systems of linear equations.
1712. u1712 Unfortunately, examination of these constructions reveals that they, too, require treating the input as a string of bits rather than of finite field elements. So for example, the Naor-Reingold construction involves modular exponentiation, which of course goes outside the arithmetic circuit model, where only addition and multiplication are allowed.
1713. u1713 At this point I can’t resist stating my own opinion, which is that the issue here is partly technical but also partly social. Simply put: Naor-Reingold and Banerjee et al. are taken to be relevant to natural proofs, because factoring, discrete logarithm, and solving noisy systems of linear equations have become accepted by the community of cryptographers as plausibly hard problems. Since real computers use Boolean circuits, and since in practice one normally needs pseudorandom functions rather than polynomials, cryptographers have had extremely little reason to study pseudorandom low-degree polynomials that are computed by small arithmetic circuits over finite fields.
1714. u1714 If they had studied that, though, it seems plausible that they would’ve found decent candidates for such polynomials, and formed a consensus that they indeed seem hard to distinguish from random polynomials.
1715. u1715 Motivated by that thought, in a 2008 blog post [5], I offered my own candidate for a pseudorandom family of polynomials, ps : Fn →F, which are homogeneous of degree d = nO(1).
1716. u1716 My candidate was simply this: motivated by Valiant’s result [261] that the determinant can express any arithmetic formula (Theorem 75), take the random seed s to encode d2 uniformly-random linear functions, Li,j : Fn →F for all i, j ∈{1, . . . , d}. Then set
1717. u1717 My conjecture is that, ( at least when d is sufficiently large, a random ps drawn from this family should require exp
1718. u1718 time to distinguish from a random homogeneous polynomial of degree d, if we’re given the ( polynomial p : Fn →F by a table of |F|n values. If d is a large enough polynomial
1719. u1719 is greater than |F|O(n), so the natural proofs barrier would apply. in n, then exp
1720. u1720 So far there’s been little study of this conjecture.
1721. u1721 Neeraj Kayal and Joshua Grochow (personal communication) have pointed out to me that the Mignon-Ressayre Theorem (Theorem 74) implies that ps can be efficiently distinguished from a uniformly-random degree-d homogeneous polynomial whenever d < n/2.
1722. u1722 For Mignon and Ressayre show that the Hessian matrix, Hp (X), satisfies rank (Hp (X)) ≤2d for all X if p is a d × d determinant, whereas a random p would have rank (Hp (X)) = n for most X with overwhelming probability, so this provides a distinguisher. However, it’s not known what happens for larger d.
1723. u1723 As this survey was being completed, Grochow et al. [116] and Forbes, Shpilka, and Lee Volk [90] published the first papers that try to set up formal frameworks for arithmetic natural proofs. These authors don’t discuss concrete candidates for pseudorandom polynomials, but their frameworks could be instantiated with a candidate like our ps above. One crucial observation both papers make is that, in at least one respect, the natural proofs barrier works out better in the arithmetic setting than in the Boolean one.
1724. u1724 Specifically, suppose we restrict attention to distinguishing algorithms that themselves apply a low-degree polynomial q to the coefficient vector vp of our polynomial p, and that output “p is hard” if q (vp)̸ = 0, or “p might be simple” if q (vp) = 0.
1725. u1725 Then if p is over a field F of large enough characteristic, then we don’t need to postulate Razborov and Rudich’s “largeness” condition separately for q.
1726. u1726 We get that condition automatically, simply because a polynomial over F that’s nonzero anywhere is nonzero almost everywhere.
1727. u1727 This means that the question of whether there’s an arithmetic natural proofs barrier, at least of this new type, can be restated as a question about whether the set of “simple” polynomials p is a hitting set against the set of polynomials q. In other words, if q is nonzero, then must there exist a “simple” p such that q (vp)̸ = 0?
1728. u1728 Of course, if our goal is to prove P̸ = NP or NP̸ ⊂P/poly, then perhaps the whole question of arithmetic natural proofs is ultimately beside the point.
1729. u1729 For to prove NP̸ ⊂P/poly, we’ll need to become experts at overcoming the natural proofs barrier in any case: either in the arithmetic world, or if not, then when we move from the arithmetic world back to the Boolean one.
1730. u1730 6.6
1731. u1731 Geometric Complexity Theory I’ll end this survey with some extremely high-level remarks about Geometric Complexity Theory (GCT): an ambitious program to prove P̸ = NP and related conjectures using algebraic geometry and representation theory. This program has been pursued since the late 1990s; it was started by Ketan Mulmuley, with important contributions from Milind Sohoni and others. GCT has changed substantially since its inception: for example, as we’ll see, recent negative results by Ikenmeyer and Panova [130] among others have decisively killed certain early hopes for the GCT program, while still leaving many avenues to explore in GCT more broadly construed. Confusingly, the term “GCT” can refer either to the original Mulmuley-Sohoni vision, or to the broader interplay between complexity and algebraic geometry, which is now studied by a community of researchers, many of whom deviate from Mulmuley and Sohoni on various points. In this section, I’ll start by explaining the original Mulmuley-Sohoni perspective (even where some might consider it obsolete), and only later discuss more recent developments. Also, while the questions raised by GCT have apparently sparked quite a bit of interesting progress in pure mathematics—much of it only marginally related to complexity theory—in this section I’ll concentrate exclusively on the quest to prove circuit lower bounds.
1732. u1732 I like to describe GCT as “the string theory of computer science.” Like string theory, GCT has the aura of an intricate theoretical superstructure from the far future, impatiently being worked on today. Both have attracted interest partly because of “miraculous coincidences” (for string theory, these include anomaly cancellations and the prediction of gravitons; for GCT, exceptional properties of the permanent and determinant, and surprising algorithms to compute the multiplicities of irreps).
1733. u1733 Both have been described as deep, compelling, and even “the only game in town” (not surprisingly, a claim disputed by the fans of rival ideas!).
1734. u1734 And like with string theory, there are few parts of modern mathematics not known or believed to be relevant to GCT.
1735. u1735 For both string theory and GCT, however, the central problem has been to say something novel and verifiable about the real-world phenomena that motivated the theory in the first place, and to do so in a way that depends essentially on the theory’s core tenets (rather than on inspirations or analogies, or on small fragments of the theory).
1736. u1736 For string theory, that would mean making confirmed predictions or (e.g.) explaining the masses and generations of the elementary particles; for GCT, it would mean proving new circuit lower bounds.
1737. u1737 Indeed, proponents of both theories freely admit that one might spend one’s whole career on the theory, without living to see a payoff of that kind.85
1738. u1738 GCT is not an easy subject, and I don’t pretend to be an expert.
1739. u1739 Part of the difficulty is inherent, while part of it is that the primary literature on GCT is not optimized for beginners: it contains a profusion of ideas, extended analogies, speculations, theorems, speculations given
1740. u1740 85Furthermore, just as string theory didn’t predict what new data there has been in fundamental physics in recent decades (e.g., the dark energy), so GCT played no role in, e.g., the proof of NEXP̸ ⊂ACC (Section 6.4.2), or the breakthrough lower bounds for small-depth circuits computing the permanent (Section 6.5.2).
1741. u1741 In both cases, to point this out is to hold the theory to a high and possibly unfair standard, but also ipso facto to pay the theory a compliment.
1742. u1742 acronyms and then referred to as if they were theorems, and advanced mathematics assumed as background.
1743. u1743 Mulmuley regards the beginning of GCT as a 1997 paper of his [188], which used algebraic geometry to prove a circuit lower bound for the classic MaxFlow problem. In MaxFlow, we’re given as input a description of an n-vertex directed graph G, with nonnegative integer weights on the edges (called capacities), along with designated source and sink vertices s and t, and a nonnegative integer w (the target).
1744. u1744 The problem is to decide whether w units of a liquid can be routed from s to t, with the amount of liquid flowing along an edge e never exceeding e’s capacity. The MaxFlow problem is famously in P,86 but the known algorithms are inherently serial, and it remains open whether they can be parallelized.
1745. u1745 More concretely, is MaxFlow in NC1, or
1746. u1746 Note that any proof of MaxFlow /∈NC1 would imply the some other small-depth circuit class?
1747. u1747 spectacular separation NC1̸ = P.
1748. u1748 Nevertheless, Mulmuley [188] managed to prove a strong lower bound for a restricted class of circuits, which captures almost all the MaxFlow algorithms known in practice.
1749. u1749 Theorem 88 (Mulmuley [188]) Consider an arithmetic circuit for MaxFlow, which takes as input the integer capacities and the integer target w.87
1750. u1750 The gates, which all have fanin 2, can perform integer addition, subtraction, and multiplication (+, −, ×) as well as integer comparisons (=, ≤, <).
1751. u1751 A comparison gate returns the integer 1 if it evaluates to “true” and 0 if it evaluates to “false.” The circuit’s final output must be 1 if the answer is “yes” and 0 if the answer is “no.” Direct access to the bit-representations of the integers is not allowed, nor (for example) are the floor and ceiling functions.
1752. u1752 Any and such ceiling circuit functions. ( must have depth Ω(√n).
1753. u1753 Moreover, this holds even if the capacities are restricted to be O
1754. u1754 Similar to previous arithmetic complexity lower bounds, the proof of Theorem 88 proceeds by noticing that any small-depth arithmetic circuit separates the yes-instances from the no-instances via a small number of low-degree algebraic surfaces. It then appeals to results from algebraic geometry (e.g., the Milnor-Thom Theorem) to show that, for problems of interest (such as MaxFlow), no such separation by low-degree surfaces is possible. As Mulmuley already observed in [188], the second part of the argument would work just as well for a random problem.
1755. u1755 Since polynomial degree is efficiently computable, this means that we get a natural proof in the Razborov-Rudich sense (Section 6.2.5). So the same technique can’t possibly work to prove NC1̸ = P: it must break down when arbitrary bit operations are allowed.
1756. u1756 Nevertheless, for Mulmuley, Theorem 88 was strong evidence that algebraic geometry provides a route to separating complexity classes.
1757. u1757 The foundational GCT papers by Mulmuley and his collaborators, written between 2001 and 2013, are “GCT1” through “GCT8” [197, 198, 196, 50, 195, 192, 192, 189, 190], though much of the material in those papers has been rendered obsolete by subsequent developments. Readers seeking a less overwhelming introduction could try Mulmuley’s overviews in Communications of the ACM [194] or in Journal of the ACM [193]; or lecture notes and other materials on Mulmuley’s GCT
1758. u1758 86MaxFlow can easily be reduced to linear programming, but Ford and Fulkerson [91] also gave a much faster and direct way to solve it, which can be found in any undergraduate algorithms textbook. There have been further improvements since.
1759. u1759 87We can assume, if we like, that G, s, and t are hardwired into the circuit.
1760. u1760 We can also allow constants such as 0 and 1 as inputs, but this is not necessary, as we can also generate constants ourselves using comparison gates and arithmetic.
1761. u1761 website [187]; or surveys by Regan [224] or by Landsberg [162]. Perhaps the most beginner-friendly exposition of GCT yet written—though it covers only the early parts of the program—is contained in Joshua Grochow’s PhD thesis [114, Chapter 3].
1762. u1762 6.6.1
1763. u1763 From Complexity to Algebraic Geometry So what is GCT?
1764. u1764 It’s easiest to understand GCT as a program to prove Valiant’s Conjecture 72—that is, to show that any affine embedding of the n × n permanent over C into the m × m determinant over C requires (say) m = 2nΩ(1), and hence, that the permanent requires exponentialsize arithmetic circuits.
1765. u1765 GCT also includes an even more speculative program to prove Boolean lower bounds, such as NP̸ ⊂P/poly and hence P̸ = NP.
1766. u1766 However, if we accept the premises of GCT in the first place (e.g., the primacy of algebraic geometry for circuit lower bounds), then we might as well start with the permanent versus determinant problem, since that’s where the core ideas of GCT come through the most clearly.
1767. u1767 The first observation Mulmuley and Sohoni make is that Valiant’s Conjecture 72 can be translated into an algebraic-geometry conjecture about orbit closures. In more detail, consider a group G that acts on a set V : in the examples that interest us, G will be a group of complex matrices, while V will be a high-dimensional vector space over C (e.g., the space of all homogeneous degree-n polynomials over some set of variables).
1768. u1768 Then the orbit of a point v ∈V , denoted Gv, is the set {g · v : g ∈G}. Also, the orbit closure of x, denoted Gv, is the closure of Gv in the usual complex topology. It contains all the points that can be arbitrarily well approximated by points in Gv.
1769. u1769 To be more concrete, let G = GLm2 (C) be the general linear group: the group of all invertible m2 × m2 complex matrices. Also, let V be the vector space of all homogeneous degree-m complex polynomials over m2 variables; the variables are labeled x1,1, . . . , xm,m, and we’ll think of them as the entries of an m × m matrix X.
1770. u1770 (Recall that a homogeneous polynomial is one where all the monomials have the same degree m; restricting to them lets GCT talk about linear maps rather
1771. u1771 Then G acts on V in an obvious way: for all matrices A ∈G and polynomials than affine ones.) p ∈V , we can set (A · p) (x) := p (Ax).
1772. u1772 Indeed, this action is a group representation: each A ∈G acts linearly on the coefficients of p, so we get a homomorphism from G to the linear transformations on V .
1773. u1773 Now, we’d like to interpret both the m × m determinant and the n × n permanent (n ≪m) as points in V , in order to phrase the permanent versus determinant problem in terms of orbit closures.
1774. u1774 For the determinant, this is trivial: Detm is a degree-m homogeneous polynomial over the xij’s.
1775. u1775 The n × n permanent, on the other hand, is a lower-degree polynomial over a smaller set of variables. GCT solves that problem by considering the so-called padded permanent,
1776. u1776 where X|n denotes the top-left n × n submatrix of X, and xm,m is just some entry of X that’s not in X|n. This is a homogeneous polynomial of degree m.
1777. u1777 Let χDet,m := G · Detm be the orbit closure of the m × m determinant, and let χPer,m,n := G · Per∗
1778. u1778 m,n be the orbit closure of the padded n × n permanent.
1779. u1779 I can now state the central conjecture that GCT seeks to prove.
1780. u1780 Conjecture 89 (Mulmuley-Sohoni Conjecture) If m = 2no(1), then for all sufficiently large n we have Per∗
1781. u1781 m,n /∈χDet,m, or equivalently χPer,m,n̸ ⊂χDet,m. In other words, the padded permanent is not in the orbit closure of the determinant.
1782. u1782 Mulmuley and Sohoni’s first observation is that a proof of Conjecture 89 (or indeed, any lower bound on m better than nΩ(log n)) would imply a proof of Valiant’s Conjecture: Proposition 90 ([197]) Suppose there’s an affine embedding of the n × n permanent into the m × m determinant: i.e., D (n) ≤m, in the notation of Section 6.5.1.
1783. u1783 Let me make two remarks about Proposition 90. First, for the proposition to hold, it’s crucial that we’re talking about the orbit closure, not just the orbit.
1784. u1784 m,n /∈ G · Detm—for example, because every element of G · Detm is irreducible (can’t be factored into lower-degree polynomials), whereas the padded permanent is clearly reducible.
1785. u1785 But that tells us only that there’s no invertible linear transformation of the variables that turns the determinant into the padded permanent, not that there’s no linear transformation at all.
1786. u1786 In GCT, linear changes of variable play the role of reductions—so, while the orbit of f plays the role of the “f-complete problems,” it’s the orbit closure that plays the role of the complexity class of functions reducible to f.88
1787. u1787 The second remark is that, despite their similarity, it’s unknown whether Conjecture 89 is equivalent to Valiant’s conjecture that the permanent requires affine embeddings into the determinant of size D (n) > 2no(1).
1788. u1788 The reason is that there are points in the orbit closure of the determinant that aren’t in its “endomorphism orbit” (that is, the set of polynomials that have not-necessarilyinvertible linear embeddings into the determinant). In complexity terms, these are homogeneous degree-m polynomials that can be arbitrarily well approximated by determinants of m×m matrices of linear functions, but not represented exactly.
1789. u1789 See Grochow [114] for further discussion of both issues.
1790. u1790 6.6.2
1791. u1791 Characterization by Symmetries So far, it seems like all we’ve done is restated Valiant’s Conjecture in a more abstract language and slightly generalized it.
1792. u1792 But now we come to the main insight of GCT, which is that the permanent and determinant are both special, highly symmetric functions, and it’s plausible that we can leverage that fact to learn more about their orbit closures than we could if they were arbitrary functions.
1793. u1793 For starters, Per (X) is symmetric under permuting X’s rows or columns, transposing X, and multiplying the rows or columns by scalars that multiply to 1.
1794. u1794 That is, we have
1795. u1795 (2) for all permutation matrices P and Q, and all diagonal matrices A and B such that Per (A) Per (B) = 1. The determinant has an even larger symmetry group: we have
1796. u1796 (3)
1797. u1797 88More broadly, there are many confusing points about GCT whose resolutions require reminding ourselves that we’re talking about orbit closures, not only about orbits.
1798. u1798 For example, it’s often said that the plan of GCT is ultimately to show that the n × n permanent has “too little symmetry” to be embedded into the m × m determinant, unless m is much larger than n. But in that case, what about (say) a random arithmetic formula f of size n, which has no nontrivial symmetries, but which clearly can be embedded into the (n + 1)×(n + 1) determinant, by Theorem 75?
1799. u1799 Even though this f clearly isn’t characterized by its symmetries, mustn’t the embedding obstructions for f be a strict superset of the embedding obstructions for the permanent—since f’s symmetries are a strict subset of the permanent’s symmetries—and doesn’t that give rise to a contradiction?
1800. u1800 The solution to the apparent paradox is that this argument would be valid if we were talking only about orbits, but it’s not valid for orbit closures.
1801. u1801 With orbit closures, the set of obstructions doesn’t depend in a simple way on the symmetries of the original function, so it’s possible that an obstruction for the permanent would fail to be an obstruction for f.
1802. u1802 for all matrices A and B such that Det (A) Det (B) = 1.
1803. u1803 But there’s a further point: it turns out that the permanent and determinant are both uniquely characterized (up to a constant factor) by their symmetries, among all homogeneous polynomials of the same degree. More precisely: Theorem 91 Let p be any degree-m homogeneous polynomial in the entries of X ∈Cm×m that satisfies p (X) = p (PXQ) = p (AXB) for all permutation matrices P, Q and diagonal A, B with Per (A) Per (B) = 1.
1804. u1804 Then p (X) = α Per (X) for some α ∈C.
1805. u1805 Likewise, let p be any degree-m homogeneous polynomial in the entries of X ∈Cm×m that satisfies p (X) = p (AXB) for all A, B with Det (A) Det (B) = 1.
1806. u1806 Then p (X) = α Det (X) for some α ∈C.89
1807. u1807 Theorem 91 is fairly well-known in representation theory; the determinant case dates back to Frobenius. See Grochow [114, Propositions 3.4.3 and 3.4.5] for an elementary proof, using Gaussian elimination for the determinant and even simpler considerations for the permanent.
1808. u1808 Notice that we’re not merely saying that any polynomial p with the same symmetry group as the permanent is a multiple of the permanent (and similarly for the determinant), but rather that any p whose symmetry group contains the permanent’s is a multiple of the permanent.
1809. u1809 In a sense, Theorem 91 is the linchpin of the GCT program.
1810. u1810 Among other things, it’s GCT’s answer to the question of how it could overcome the natural proofs barrier. For notice that, if we picked a degree-m homogeneous polynomial at random, it almost certainly wouldn’t be uniquely characterized by its symmetries, as the permanent and determinant are.90
1811. u1811 Thus, if a proof that the permanent is hard relies on symmetry-characterization, we need not fear that the same proof would work for a random homogeneous polynomial, and thereby give us a way to break arithmetic pseudorandom functions (Section 6.5.3).
1812. u1812 While this isn’t mentioned as often, Theorem 91 should also let GCT overcome the relativization and algebrization barriers, since (for example) a polynomial that was #PA-complete for some oracle A, rather than #P-complete like the permanent was, wouldn’t have the same symmetries as the permanent itself.
1813. u1813 6.6.3
1814. u1814 The Quest for Obstructions Because the permanent and determinant are characterized by their symmetries, and because they satisfy another technical property called “partial stability,” Mulmuley and Sohoni observe that a field called geometric invariant theory can be used to get a handle on their orbit closures. I won’t explain the details of how this works (which involve something called Luna’s ´Etale Slice Theorem [177]), but will just state the punchline.
1815. u1815 Given a set S ⊆CN, define R [S], or the coordinate ring of S, to be the vector space of all complex polynomials q : CN →C, with two polynomials identified if they agree on all points x ∈S. Then we’ll be interested in RDet := R [χDet,m] and RPer := R [χPer,m,n]: the coordinate rings ) of the orbit closures of the determinant and the padded permanent.
1816. u1816 is the dimension of the vector space of homogeneous degree-m polynomials over m2 variables. m
1817. u1817 So the coordinate rings are vector spaces of polynomials over N variables: truly enormous objects.
1818. u1818 89Note that we don’t even need to assume the symmetry p (X) = p
1819. u1819 Also, it might seem like “cheating” that we use the permanent to state the symmetries that characterize the permanent, and likewise for the determinant. But we’re just using the permanent and determinant as convenient ways to specify which matrices A, B we want, and could give slightly more awkward symmetry conditions that avoided them. (This is especially clear for the permanent, since if A is diagonal, then Per (A) is just the product of the diagonal entries.)
1820. u1820 90See Grochow [114, Proposition 3.4.9] for a simple proof of this, via a dimension argument.
1821. u1821 Next, let q : CN →C be one of these “big” polynomials, whose inputs are the coefficients of a “small” polynomial p (such as the permanent or determinant). Then we can define an action of the general linear group, G = GLm2 (C), on q, via (A · q) (p (x)) := q (p (Ax)) for all A ∈G.
1822. u1822 In other words, we take the action of G on the “small” polynomials p that we previously defined, and use it to induce an action on the “big” polynomials q. Notice that this action fixes the coordinate rings RDet and RPer (i.e., just shifts their points around), simply because the action of G fixes the orbit closures χDet,m and χPer,m,n themselves.
1823. u1823 As a consequence, the actions of G on RDet and RPer give us two representations of the group G: that is, homomorphisms that map the elements of G to linear transformations on the vector spaces RDet and RPer respectively. Call these representations ρDet and ρPer respectively.
1824. u1824 Like most representations, ρDet and ρPer can be decomposed uniquely into direct sums of isotypic components; each isotypic component, in turn, consists of an irreducible representation, or “irrep” (which can’t be further decomposed) that occurs with some nonnegative integer multiplicity.91
1825. u1825 In particular, let ρ : G →Ck×k be any irrep of G.
1826. u1826 Then ρ occurs with some multiplicity, call it λDet (ρ), in ρDet, and with some possibly different multiplicity, call it λPer (ρ), in ρPer. We’re now ready for the theorem that sets the stage for the rest of GCT.
1827. u1827 Theorem 92 (Mulmuley-Sohoni [197]) Suppose there exists an irrep ρ such that λPer (ρ) > λDet (ρ).
1828. u1828 m,n /∈χDet,m: that is, the padded permanent is not in the orbit closure of the determinant.
1829. u1829 Note that Theorem 92 is not an “if and only if”: even if Per∗
1830. u1830 m,n /∈χDet,m, there’s no result saying that the reason must be representation-theoretic.
1831. u1831 In GCT2 [198], Mulmuley and Sohoni conjecture that the algebraic geometry of χDet,m is in some sense completely determined by its representation theory, but if true, that would have to be for reasons specific to χDet,m (or other “complexity-theoretic” orbit closures).
1832. u1832 If λPer (ρ) > λDet (ρ), then Mulmuley and Sohoni call ρ a multiplicity obstruction to embedding the permanent into the determinant.
1833. u1833 Any obstruction would be a witness to the permanent’s hardness: in crude terms, it would prove that the m × m determinant has “the wrong kinds of symmetries” to express the padded n × n permanent, unless m is much larger than n.
1834. u1834 From this point forward, GCT—at least in Mulmuley and Sohoni’s vision—is focused entirely on the hunt for a multiplicity obstruction.
1835. u1835 A priori, one could imagine proving nonconstructively that an obstruction ρ must exist, without actually finding it. However, Mulmuley and Sohoni emphatically reject that approach. They want not merely any proof of Conjecture 89, but an “explicit” proof: that is, one that yields an algorithm that actually finds an obstruction ρ witnessing Per∗
1836. u1836 m,n /∈χDet,m, in time polynomial in m and n. Alas, as you might have gathered, the representations ρDet and ρPer are fearsomely complicated objects—so even if we accept for argument’s sake that obstructions exist, we seem a very long way from algorithms to find them in less than astronomical time.92
1837. u1837 For now, therefore, Mulmuley and Sohoni argue that the best way to make progress toward Conjecture 89 is to work on more and more efficient algorithms to compute the multiplicities of
1838. u1838 91An isotypic component might be decomposable into irreps in many ways, but one always gets the same number of irreps of the same type.
1839. u1839 92In principle, ρDet and ρPer are infinite-dimensional representations, so an algorithm could search them forever for obstructions without halting. On the other hand, if we impose some upper bound on the degrees of the polynomials in the coordinate ring, we get an algorithm that takes “merely” doubly- or triply-exponential time.
1840. u1840 irreps in complicated representations like ρDet and ρPer. The hope is that, in order to design those algorithms, we’ll be forced to acquire such a deep understanding that we’ll then know exactly where to look for a ρ such that λPer (ρ) > λDet (ρ).
1841. u1841 So that’s the program that’s been pursued for the last decade; I’ll have more to say later about where that program currently stands.
1842. u1842 The central idea here—that the path to proving P̸ = NP will go through discovering new algorithms, rather than through ruling them out—is GCT’s version of “ironic complexity theory,” discussed in Section 6.4. What I’ve been calling “irony” in this survey, Mulmuley calls “The Flip” [191]: that is, flipping lower-bound problems into upper-bound problems, which we have a much better chance of solving.
1843. u1843 Stepping back from the specifics of GCT, Mulmuley’s view is that, before we prove (say) NP̸ ⊂ P/poly, a natural intermediate goal is ( to find an algorithm A that takes a positive integer n as input, runs for nO(1) time (or even exp
1844. u1844 time), and then outputs a proof that 3Sat instances of size n have no circuits of size m, for some superpolynomial function m.
1845. u1845 Such an algorithm wouldn’t immediately prove NP̸ ⊂P/poly, because we might still not know how to prove that A succeeded for every n.
1846. u1846 Even so, it would clearly be a titanic step forward, since we could run A and check that it did succeed for every n we chose, perhaps even for n’s in the billions.
1847. u1847 At that point, we could say either that NP̸ ⊂P/poly, or else that NP ⊂P/poly only “kicks in” at such large values of n as to have few or no practical consequences. Furthermore, Mulmuley argues, we’d then be in a much better position to prove NP̸ ⊂P/poly outright, since we’d “merely” have to analyze A, whose very existence would obviously encode enormous insight about the problem, and prove that it worked for all n.93
1848. u1848 There has indeed been progress in finding efficient algorithms to compute the multiplicities of irreps, though the state-of-the-art is still extremely far from what GCT would need.
1849. u1849 To give an example, a Littlewood-Richardson coefficient is the multiplicity of a given irrep in a tensor product of two irreps of the general linear group GLn (C).
1850. u1850 In GCT3 [196], Mulmuley et al. observed that a result of Knutson and Tao [156] implies that one can use linear programming, not necessarily to compute Littlewood-Richardson coefficients in polynomial time, but at least to decide whether they’re positive or zero.94 B¨urgisser and Ikenmeyer [66] later gave a faster polynomial-time algorithm for the same problem; theirs was purely combinatorial and based on maximum flow.
1851. u1851 Note that, even if we only had efficient algorithms for positivity, those would still be useful for finding so-called occurrence obstructions: that is, irreps ρ such that λPer (ρ) > 0 even though λDet (ρ) = 0. Thus, the “best-case scenario” for GCT would be for the permanent’s hardness to be witnessed not just by any obstructions, but by occurrence obstructions.
1852. u1852 As we’ll see, in a recent breakthrough, this “best-case scenario” has been ruled out [130, 68].
1853. u1853 In many cases, we don’t even know yet how to represent the multiplicity of an irrep as a #P function: at best, we can represent it as a difference between two #P functions.
1854. u1854 In those cases, research effort in GCT has sought to give “positive formulas” for the multiplicities: in other words, to represent them as sums of exponentially many nonnegative terms, and thereby place their computation in #P itself.
1855. u1855 The hope is that a #P formula could be a first step toward a
1856. u1856 93A very loose analogy: well before Andrew Wiles proved Fermat’s Last Theorem [270], that xn + yn = zn has no nontrivial integer solutions for any n ≥3, number theorists knew a reasonably efficient algorithm that took an exponent n as input, and that (in practice, in all the cases that were tried) proved FLT for that n.
1857. u1857 Using that algorithm, in 1993—just before Wiles announced his proof—Buhler et al. [61] proved FLT for all n up to 4 million.
1858. u1858 94As pointed out in Section 2.2.6, there are other cases in complexity theory where deciding positivity is much easier than counting: for example, deciding whether a graph has at least one perfect matching (counting the number of perfect matchings is #P-complete).
1859. u1859 polynomial-time algorithm to decide positivity. To illustrate, Blasiak et al. gave a #P formula for “two-row Kronecker coefficients” in GCT4 [50], while Ikenmeyer, Mulmuley, and Walter [129] did so for a different subclass of Kronecker coefficients.
1860. u1860 6.6.4 Suppose—let’s dream—that everything above worked out perfectly. That is, suppose GCT led to the discovery of explicit obstructions for embedding the padded permanent into the determinant, and thence to a proof of Valiant’s Conjecture 72.
1861. u1861 How would GCT go even further, to prove P̸ = NP?
1862. u1862 The short answer is that Mulmuley and Sohoni [197] defined an NP function called E, as well as a P-complete95 function called H, and showed them to be characterized by symmetries in only a slightly weaker sense than the permanent and determinant are.
1863. u1863 The E and H functions aren’t nearly as natural as the permanent and determinant, but they suffice to show that P̸ = NP could in principle be proven by finding explicit representation-theoretic obstructions, which in this case would be representations associated with the orbit of E but not with the orbit of H. Alas, because E and H are functions over a finite field Fq rather than over C, the relevant algebraic geometry and representation theory would all be over finite fields as well. This leads to mathematical questions even less well-understood (!) than the ones discussed earlier, providing some additional support for the intuition that proving P̸ = NP should be “even harder” than proving Valiant’s Conjecture.
1864. u1864 For illustration, let me now define Mulmuley and Sohoni’s E function. Let X0 and X1 be two n × n matrices over the finite field Fq.
1865. u1865 Also, given a binary string s = s1 · · · sn, let Xs be the n × n matrix obtained by choosing the ith column from X0 if si = 0 or from X1 if si = 1 for all i ∈{1, . . . , n}. We then set
1866. u1866 and finally set
1867. u1867 to obtain a function in {0, 1}. Testing whether E (X0, X1) = 1 is clearly an NP problem, since an NP witness is a single s such that Det (Xs) = 0. Furthermore, Gurvits [120] showed that, at least over Z, testing whether F (X0, X1) = 0 is NP-complete.
1868. u1868 Interestingly, it’s not known whether E itself is NP-complete—though of course, to prove P̸ = NP, it would suffice to put any NP problem outside P, not necessarily an NP-complete one.
1869. u1869 The main result about E (or rather F) is the following:
1870. u1870 Theorem 93 (see Grochow [114, Proposition 3.4.6]) Let p : F2n2 q
1871. u1871 →Fq be any homogeneous, degree-n2n polynomial in the entries of X0 and X1 that’s divisible by Det q (X0) Det (X1), and suppose every linear symmetry of F is also a linear symmetry of p.
1872. u1872 Then p (X0, X1) = αF (X0, X1) for some α ∈Fq.
1873. u1873 The proof of Theorem 93 involves some basic algebraic geometry.
1874. u1874 Even setting aside GCT, it would be interesting to know whether the existence of a plausibly-hard NP problem that’s characterized by its symmetries had direct complexity-theoretic applications.
1875. u1875 95A language L is called P-complete if (1) L ∈P, and (2) every L′ ∈P can be reduced to L by some form of reduction weaker than arbitrary polynomial-time ones (LOGSPACE reductions are often used for this purpose).
1876. u1876 One last remark: for some complexity classes, such as BQP, we currently lack candidate problems characterized by their symmetries, so even by the speculative standards of this section, it’s unclear how GCT could be used to prove (say) P̸ = BQP or NP̸ ⊂BQP.
1877. u1877 6.6.5
1878. u1878 Reports from the Trenches In the past few years, there’s been a surprising amount of progress on resolving the truth or falsehood of some of GCT’s main hypotheses, and on relating GCT to mainstream complexity theory. This section relays some highlights from the rapidly-evolving story.
1879. u1879 Alas, the news has been sharply negative for the “simplest” way GCT could have worked: namely, by finding occurrence obstructions.
1880. u1880 In a 2016 breakthrough, B¨urgisser, Ikenmeyer, and Panova [68], building on slightly earlier work by Ikenmeyer and Panova [130], showed that one can’t use occurrence obstructions to separate χPer,m,n from χDet,m, for m superpolynomially larger than n.
1881. u1881 That is, any irrep that occurs at least once in the padded permanent representation ρPer, also occurs at least once in the determinant representation ρDet.
1882. u1882 Particularly noteworthy is one aspect of the proof of this result.
1883. u1883 The authors assume by contradiction that an occurrence obstruction exists, to separate χPer,m,n from χDet,m for some fixed n and m. They then show inductively that there would also be obstructions proving χPer,m,n̸ ⊂χDet,m for larger and larger m—until finally, one would have a contradiction with (for example) Theorem 73, the result of Grenet [111] showing that χPer,2n−1,n ⊆χDet,2n−1.
1884. u1884 This underscores one of the central worries about GCT since the beginning: namely, how is this approach sensitive to the quantitative nature of the permanent versus determinant problem?
1885. u1885 What is it that would break down, if one tried to use GCT to prove lower bounds so aggressive that they were actually false? In the case of occurrence obstructions, the answer turns out to be that nothing would break down.
1886. u1886 On the positive side, Ikenmeyer, Mulmuley, and Walter [129] showed that there are superpolynomially many Kronecker coefficients that do vanish, thereby raising hope that occurrence obstructions might exist after all, if not for permanent versus determinant then perhaps for other problems.
1887. u1887 Notably, they proved this result by first giving a #P formula for the relevant class of Kronecker coefficients, thereby illustrating the GCT strategy of first looking for algorithms and only later looking for the obstructions themselves.
1888. u1888 Also on the positive side, Landsberg and Ressayre [165] recently showed that Theorem 73— the embedding of n × n permanents into (2n −1) × (2n −1) determinants due to Grenet [111]—is exactly optimal among all embeddings that respect left-multiplication by permutation and diagonal matrices (thus, roughly half the symmetry group of the n × n permanent), as Grenet’s embedding turns out to do.
1889. u1889 We also now know that the central property of the permanent and determinant that GCT seeks to exploit—namely, their characterization by symmetries—does indeed have complexitytheoretic applications.
1890. u1890 In particular, Mulmuley [191] observed that one can use the symmetrycharacterization of the permanent to give a different, and in some ways nicer, proof of the classic result of Lipton [173] that the permanent is self-testable: that is, given a circuit C that’s alleged to compute Per (X), in randomized polynomial time one can either verify that C (X) = Per (X) for most matrices X, or else find a counterexample where C (X)̸ = Per (X).96
1891. u1891 Subsequently, Kayal [147] took Mulmuley’s observation further to prove the following.
1892. u1892 96This result of Lipton’s provided the germ of the proof that IP = PSPACE; see Section 6.3.1.
1893. u1893 Mulmuley’s test improves over Lipton’s by, for example, requiring only nonadaptive queries to C rather than adaptive ones.
1894. u1894 Theorem 94 (Kayal [147]) Let n = d2, and suppose we’re given black-box access to a degreed polynomial p : Fn →F, for some field F.
1895. u1895 Suppose also that we’re promised that p (X) = Per (L (X)), where L is a d × d matrix of affine forms in the variables X = (x1, . . . , xn), and the mapping X →L (X) is invertible (that is, has rank n).
1896. u1896 Then there’s a randomized algorithm, running in nO(1) time, that actually finds an L such that p (X) = Per (L (X)).
1897. u1897 The same holds if p (X) = Det (L (X)).
1898. u1898 In the same paper [147], Kayal showed that if q is an arbitrary polynomial, then deciding whether there exist A and b such that p (x) = q (Ax + b) is NP-hard.
1899. u1899 So what is it about the permanent and determinant that made the problem so much easier?
1900. u1900 The answer turns out to be their symmetry-characterization, along with more specific properties of the Lie algebras of the stabilizer groups of Per and Det.
1901. u1901 For more about the algorithmic consequences of symmetry-characterization, see for example Grochow [114, Chapter 4], who also partially derandomized Kayal’s algorithm and applied it to other problems such as matrix multiplication.
1902. u1902 In my view, an exciting challenge is to use the symmetry-characterization of the permanent, or perhaps of the E function from Section 6.6.4, to prove other new complexity results—not necessarily circuit lower bounds—that hopefully evade the relativization and algebrization barriers.
1903. u1903 We also now know that nontrivial circuit lower bounds—albeit, not state-of-the-art ones—can indeed be proved by finding representation-theoretic obstructions, as GCT proposed. Recall from Section 6 that the rank of a 3-dimensional tensor A ∈Fn×n×n is the smallest r such that A can be written as the sum of r rank-one tensors, tijk = xiyjzk. If F is a continuous field like C, then we can also define the border rank of A to be the smallest r such that A can be written as the limit of rank-r tensors. It’s known that border rank can be strictly less than rank.97 Border rank was introduced in 1980 by Bini, Lotti, and Romani [49] to study matrix multiplication algorithms: indeed, one could call that the first appearance of orbit closures in computational complexity, decades before GCT.
1904. u1904 More concretely, the n × n matrix multiplication tensor, say over C, is defined to be the 3dimensional tensor Mn ∈Cn2×n2×n2 whose (i, j) , (j, k) , (i, k) entries are all 1, and whose remaining n6 −n3 entries are all 0. In 1973, Strassen [251] proved a key result connecting the rank of Mn to the complexity of matrix multiplication: Theorem 95 (Strassen [251]) The rank of Mn equals the minimum number of nonscalar multiplications in any arithmetic circuit that multiplies two n × n matrices.
1905. u1905 As an immediate corollary, the border rank of Mn lower-bounds the arithmetic circuit complexity of n × n matrix multiplication. Bini [48] showed, moreover, that the exponent ω of matrix multiplication is the same if calculated using rank or border rank—so in that sense, the two are asymptotically equal.
1906. u1906 In a tour-de-force, in 2004 Landsberg [161] proved that the border rank of M2 is 7, using differential geometry methods. A corollary was that any procedure to multiply two 2 × 2 matrices requires at least 7 nonscalar multiplications, precisely matching the upper bound discovered by Strassen [250] in 1969. (The trivial upper bound is 8 multiplications.) More recently, Hauenstein,
1907. u1907 97A standard example is the 2 × 2 × 2 tensor whose (2, 1, 1), (1, 2, 1), and (1, 1, 2) entries are all 1, and whose 5 remaining entries are all 0.
1908. u1908 One can check that this tensor has a rank of 3 but border rank of 2.
1909. u1909 Ikenmeyer, and Landsberg [127] reproved the “border rank is 7” result, using methods that are more explicit and closer to GCT (but that still don’t yield multiplicity obstructions).
1910. u1910 Then, in 2013, B¨urgisser and Ikenmeyer [67] proved a border rank lower bound that did yield representation-theoretic embedding obstructions.
1911. u1911 Theorem 96 (B¨urgisser and Ikenmeyer [67]) There are representation-theoretic (GCT) occurrence obstructions that witness that the border rank of Mn is at least 3
1912. u1912 By comparison, the state-of-the-art lower bound on the border rank of matrix multiplication— but without multiplicity or occurrence obstructions—is the following.
1913. u1913 Theorem 97 (Landsberg and Ottaviani [164]) The border rank of Mn is at least 2n2 −n.99
1914. u1914 Since both lower bounds are still quadratic, neither of them shows that matrix multiplication requires more than ∼n2 time, but it’s interesting to see what bound one can currently achieve when one insists on GCT obstructions, compared to when one doesn’t insist on them.100
1915. u1915 Meanwhile, in a 2014 paper, Grochow [115] has convincingly argued that most known circuit lower bounds (though not all of them) can be put into a “broadly GCT-like” format. In particular, the AC0 and AC0 [p] lower bounds of Sections 6.2.3 and 6.2.4, the embedding lower bound of Mignon and Ressayre (Theorem 74), the lower bounds for small-depth arithmetic circuits and multilinear formulas of Section 6.5.2, and many other results can each be seen as constructing a separating module: that is, a “big polynomial” that takes as input the coefficients of an input polynomial, that vanishes for all polynomials in some complexity class C, but that doesn’t vanish for a polynomial q for which we’re proving that q /∈C.
1916. u1916 It’s important to understand that Grochow didn’t show that the known circuit lower bounds yield representation-theoretic obstructions, only that they yield separating modules.
1917. u1917 Thus, he showed that these results fit into the “GCT program” under a very broad definition, but not under a narrower definition.
1918. u1918 Interestingly, the lower bounds that don’t fit into the separating module format—such as MAEXP̸ ⊂P/poly (Theorem 56) and NEXP̸ ⊂ACC (Theorem 66)—essentially all use diagonalization as a key ingredient.
1919. u1919 From the beginning, this was understood to be a major limitation of the GCT framework: that it seems unable to capture diagonalization.
1920. u1920 A related point is that GCT, as it stands, has no way to take advantage of uniformity: for example, no way to prove P̸ = NP, without also proving the stronger result NP̸ ⊂P/poly. However, given that we can prove P̸ = EXP but can’t even prove NEXP̸ ⊂TC0 (where TC0 means the nonuniform class), it seems conceivable that uniformity could help in proving P̸ = NP.
1921. u1921 98On the other hand, B¨urgisser and Ikenmeyer [67] also showed that this is essentially the best lower bound achievable using their techniques.
1922. u1922 99Recently, Landsberg and Michalek [163] improved this slightly, to 2n2 −⌈log2 n⌉−1.
1923. u1923 100Of course, this situation raises the possibility that even if representation-theoretic obstructions do exist, proving their existence could be even harder than proving complexity class separations in some more direct way.
1924. u1924 One possible “failure mode” for GCT is that, after decades of struggle, mathematicians and computer scientists finally prove Valiant’s Conjecture and P̸ = NP—and then, after further decades of struggle, it’s shown that GCT could’ve proven these results as well (albeit with quantitatively weaker bounds).
1925. u1925 6.6.6
1926. u1926 The Lessons of GCT Expert opinion is divided about GCT’s prospects.
1927. u1927 Some feel that GCT does little more than take complicated questions and make them even more complicated—and are emboldened in their skepticism by the recent no-go results [130, 68]. Others feel that GCT is a natural and reasonable approach, and that the complication is an inevitable byproduct of finally grappling with the real issues. Of course, one can also “cheer GCT from the sidelines” without feeling prepared to work on it oneself, particularly given the unclear prospects for any computer-science payoffin the foreseeable future. (Mulmuley once told me he thought it would take a hundred years until GCT led to major complexity class separations, and he’s the optimist!)
1928. u1928 Personally, I’d call myself a qualified fan of GCT, in much the same way and for the same reasons that I’m a qualified fan of string theory.
1929. u1929 I think all complexity theorists should learn something about GCT—for one thing, because it has general lessons for the quest to prove P̸ = NP, even if it ends up not succeeding, or evolving still further away from the original Mulmuley-Sohoni vision. This section is devoted to what I believe those lessons are.
1930. u1930 A first lesson is that we can in principle evade the relativization, algebrization, and natural proofs barriers by using the existence of complete problems with special properties: as a beautiful example, the property of being “characterized by symmetries,” which the permanent and determinant both enjoy. A second lesson is that “ironic complexity theory” has even further reach than one might have thought: one could use the existence of surprisingly fast algorithms, not merely to show that certain complexity collapses would violate hierarchy theorems, but also to help find certificates that problems are hard.
1931. u1931 A third lesson is that there’s at least one route by which a circuit lower bound proof would need to know about huge swathes of “traditional, continuous” mathematics, as many computer scientists have long suspected (or feared!).
1932. u1932 But none of those lessons really gets to the core of the matter. One of the most striking features of GCT is that, even as the approach stands today, it “knows” about various nontrivial problems in P, such as maximum flow and linear programming (because they’re involved in deciding whether the multiplicities of irreps are nonzero). We knew, of course, that any proof of P̸ = NP would need to “know” that linear programming, matching, and so on are in P and are therefore different from 3Sat. (Indeed, that’s one way to express the main difficulty of the P ?= NP problem.) So the fact that GCT knows about all these polynomial-time algorithms seems reassuring. But what’s strange is that GCT seems to know the upper bounds, not the lower bounds—the power of the algorithms, but not their limitations! In other words, consider a hypothetical proof of P̸ = NP using GCT. If we ignore the details, and look from a distance of a thousand miles, the proof seems to be telling us not: “You see how weak P is? You see all these problems it can’t solve?” but rather, “You see how strong P is?
1933. u1933 You see all these amazing, nontrivial problems it can solve?” The proof would seem to building up an impressive case for the wrong side of the argument!
1934. u1934 One response would be to point out that this is math, not a courtroom debate, and leave it at that.
1935. u1935 Let A be a hypothetical problem, like matching or linear programming, that’s in P for a nontrivial reason, and that’s also definable purely in terms of its symmetries, as the permanent and determinant are.
1936. u1936 Then we can define an orbit closure χA, which captures all problems reducible to A. By assumption, χA must be contained in χP, the orbit closure corresponding to a P-complete problem, such as Mulmuley and Sohoni’s H function (see Section 6.6.4).
1937. u1937 And hence, there must not be any representation-theoretic obstruction to such a containment.
1938. u1938 In other words, if we were to compute the multiplicities m1, m2, . . . of all the irreps in the representation associated with χA, as well as the multiplicities n1, n2, . . . of the
1939. u1939 irreps associated with χP, we’d necessarily find that mi ≤ni for all i. Furthermore, by the general philosophy of GCT, once we had our long-sought explicit formulas for these multiplicities, we might well be able to use those formulas to prove the above inequalities.
1940. u1940 Now let’s further conjecture—following GCT2 [198]—that the orbit closures χA and χP are completely captured by their representation-theoretic data. In that case, by showing that there’s no representation-theoretic obstruction to χA ⊆χP, we would have proved, nonconstructively, that there exists a polynomial-time algorithm for A! And for that reason, we shouldn’t be surprised if the algorithmic techniques that are used to solve A (matching, linear programming, or whatever) have already implicitly appeared in getting to this point.
1941. u1941 Indeed, we should be worried if they didn’t appear.101
1942. u1942 More broadly, people sometimes stress that P̸ = NP is a “universal mathematical statement”: it says there’s no polynomial-time algorithm for 3Sat, no matter which area of math we use to construct such an algorithm.
1943. u1943 And thus, the argument goes, we shouldn’t be shocked if nearly every area of math ends up playing some role in the proof.
1944. u1944 It will immediately be objected that there are other “universal mathematical statements”—for example, G¨odel’s Incompleteness Theorem, or the unsolvability of the halting problem—that are nevertheless easy to prove, that don’t require anything close to every area of math. But we might make the analogy that proving the unsolvability of the halting problem is like proving that the first player has a forced win in the game of Hex, which Piet Hein invented in 1942 and John Nash independently invented in 1947 [201].102
1945. u1945 In both cases, there’s a clever gambit—diagonalization in the one case, “strategy-stealing” in the other—that lets us cut through what looks at first like a terrifyingly complex problem, and reason about it in a clean, abstract way.
1946. u1946 By contrast, proving P̸ = NP seems more like proving that White has the win in chess. Here the opening gambit fails, so (to switch metaphors) we’re thrown immediately into the deep end of the pool, into considering this way Black could win, and that one, and that one; or into considering an immense list of possible polynomial-time algorithms for 3Sat.
1947. u1947 ?= NP,
1948. u1948 Now, a natural reaction to this observation would be, not awe at the profundity of P but rather despair.
1949. u1949 Since math is infinite, and since the possible “ideas for polynomial-time algorithms” are presumably unbounded, why doesn’t the “universality” of P ?= NP mean that the task of proving could go on forever?
1950. u1950 At what point can we ever say “enough! we’ve discovered enough polynomial-time algorithms; now we’re ready to flip things around and proceed to proving P̸ = NP”?
1951. u1951 The earlier considerations about χA and χP suggest one possible answer to this question. Namely, we’re ready to stop when we’ve discovered nontrivial polynomial-time algorithms, not for all problems in P, but for all problems in P that are characterized by their symmetries.
1952. u1952 For 101It would be interesting to find a function in P, more natural than the P-complete H function, that’s completely characterized by its symmetries, and then try to understand explicitly why there’s no representation-theoretic obstruction to that function’s orbit closure being contained in χP—something we already know must be true. 102The game of Hex involves two players who take turns placing white and black stones respectively on a lattice of hexagons.
1953. u1953 White (which moves first) wins if it manages to connect the left and right sides of the lattice by a path of white stones, while Black wins if it connects the top and bottom by a path of black stones. (Note that one of the two must happen.)
1954. u1954 As John Nash observed, there’s a breathtakingly easy proof that White has a forced win in this game.
1955. u1955 Namely, suppose by contradiction that Black had the win. Then White could place a stone arbitrarily on its first move, and thereafter simulate Black’s winning strategy.
1956. u1956 The key observation is that the stone White placed in its first move will never actually hurt it: if Black’s stolen strategy calls for White to place a second stone there, then White can simply place a stone anywhere else instead.
1957. u1957 let B be a problem in P that isn’t characterized by symmetries.
1958. u1958 Then the orbit closure χB is contained in χP, and if we could prove that χB ⊆χP, then we would’ve nonconstructively shown the existence of a polynomial-time algorithm for B.
1959. u1959 But our hypothetical P̸ = NP proof doesn’t need to know about that.
1960. u1960 For since B isn’t characterized by symmetries, the GCT arguments aren’t going to be able to prove χB ⊆χP anyway.
1961. u1961 The above would seem to motivate an investigation of which functions in P (or NC1, etc.) can be characterized by their symmetries.
1962. u1962 If GCT can work at all, then the set of such functions, while presumably infinite, ought to be classifiable into a finite number of families. The speculation suggests itself that these families might roughly correspond to the different algorithmic techniques: Gaussian elimination, matching, linear programming, etc., and of course whatever other techniques haven’t yet been discovered.
1963. u1963 As a concrete first step toward these lofty visions, it would be interesting to find some example of a function in P that’s characterized by its symmetries, like the determinant is, and that’s in P only because of the existence of nontrivial polynomial-time algorithms for (say) matching or linear programming.
1964. u1964 6.6.7
1965. u1965 The Only Way?
1966. u1966 In recent years, Mulmuley has advanced the following argument [191, 194, 193]: even if GCT isn’t literally the only way forward on P ?= NP, still, the choice of GCT to go after explicit obstructions is in some sense provably unavoidable—and furthermore, GCT is the “simplest” approach to finding the explicit obstructions, so Occam’s Razor all but forces us to try GCT first.
1967. u1967 I agree that GCT represents a natural attack plan.
1968. u1968 But I disagree with the claim that we have any theorem telling us that GCT’s choices are inevitable, or “basically” inevitable. In this section, I’ll explain why.
1969. u1969 We can identify at least four major successive decisions that GCT makes:
1970. u1970 (1) To prove P̸ = NP, we should start by proving Valiant’s Conjecture 72.
1971. u1971 (2) To prove Valiant’s Conjecture, the natural approach is to prove Conjecture 89, about orbit
1972. u1972 (3) To prove Conjecture 89, the natural approach is to find explicit representation-theoretic
1973. u1973 (4) To find those obstructions, we should start by finding faster algorithms (or algorithms in
1974. u1974 lower complexity classes) to learn about the multiplicities of irreps.
1975. u1975 All four decisions are defensible, but not one is obvious. And of course, even if every proposition in a list had good probability individually (or good probability conditioned on its predecessors), their conjunction could have probability close to zero!
1976. u1976 As we saw in Section 6.5, decision (1) predates GCT by decades, so there’s no need to revisit it here.
1977. u1977 Meanwhile, decision (2) seems to involve only a small strengthening of what needs to be proved, in return for a large gain in elegance.
1978. u1978 But there’s plenty to question about decisions (3) and (4).
1979. u1979 Regarding (3): we saw, in Section 6.6.5, that even if a given embedding of orbit closures is impossible, the reason might simply not be reflected in representation theory—and even if it is, it might be harder to prove that there’s a representation-theoretic obstruction than that there’s some other obstruction, and one might get only a weaker lower bound that way.
1980. u1980 At least, that’s
1981. u1981 what seems to be true so far with the border rank of the matrix multiplication tensor.
1982. u1982 This is an especially acute concern because of the recent work of B¨urgisser, Ikenmeyer, and Panova [68], discussed earlier, which destroys the hope of finding occurrence obstructions (rather than the more general multiplicity obstructions) to separate the permanent from the determinant.
1983. u1983 Is it clear that we must, in Mulmuley’s words, “go for explicitness”: that is, look for an efficient algorithm that takes a specific n and m as input, and tries to find a witness that it’s impossible to embed the padded n × n permanent into the m × m determinant?
1984. u1984 Why not just look directly for a proof, which (if we found it) would work for arbitrary n and m = nO(1)?
1985. u1985 Mulmuley’s argument for explicitness rests on what he calls the “flip theorems” [191].
1986. u1986 These theorems, in his interpretation, assert that any successful approach to circuit lower bounds (not just GCT) will yield explicit obstructions as a byproduct. And thus, all GCT is doing is bringing into the open what any proof of Valiant’s Conjecture or NP̸ ⊂P/poly will eventually need to confront anyway.
1987. u1987 Let me now state some of the flip theorems.
1988. u1988 First, building on a 1996 learning algorithm of Bshouty et al. [60], in 2003 Fortnow, Pavan, and Sengupta showed that, if NP-complete problems are hard at all, then there must be short lists of instances that cause all small circuits to fail. Theorem 98 (Fortnow, Pavan, and Sengupta [96]) Suppose NP̸ ⊂P/poly.
1989. u1989 Then for every n and k, there’s a list of 3Sat instances φ1, . . . , φℓ, of length at most ℓ= nO(1), such that every circuit C of size at most nk fails to decide at least one φi in the list.
1990. u1990 Furthermore, such a list can be found in the class BPPNP: that is, by a probabilistic polynomial-time Turing machine with an NP oracle.103
1991. u1991 Atserias [33] showed that, in the statement of Theorem 98, we can also swap the NP oracle for the circuit C itself: in other words, the list φ1, . . . , φℓcan also be found in BPPC, in probabilistic polynomial time with a C-oracle.
1992. u1992 Likewise, if the permanent is hard, then there must be short lists of matrices that cause all small arithmetic circuits to fail—and here the lists are much easier to find than they are in the Boolean case.
1993. u1993 Theorem 99 (Mulmuley [191, 193]) Suppose Valiant’s Conjecture 72 holds (i.e., the permanent has no polynomial-size arithmetic circuits, over finite fields F with |F| ≫n).
1994. u1994 Then for every n and k, there’s a list of matrices A1, . . . , Aℓ∈Fn×n, of length at most ℓ= nO(1), such that for every arithmetic circuit C of size at most nk, there exists an i such that C (Ai)̸ = Per (Ai). Indeed, a random list A1, . . . , Aℓwill have that property with 1−o (1) probability. Furthermore, if polynomial identity testing has a black-box derandomization,104 then such a list can be found in deterministic nO(1) time.
1995. u1995 103In fact, the list can be found in the class ZPPNP, where ZPP stands for Zero-Error Probabilistic Polynomial-Time. This means that, whenever the randomized algorithm succeeds in constructing the list, it’s certain that it’s done so. (Note that referring to BPPNP and ZPPNP here is strictly an abuse of notation, since these are classes of decision problems—but theoretical computer scientists will typically perpetrate such abuses when there’s no risk of confusion.) 104The polynomial identity testing problem was defined in Section 5.4.
1996. u1996 Also, by a “black-box derandomization,” we mean a deterministic polynomial-time algorithm that outputs a hitting set: that is, a list of points x1, . . . , xℓsuch that, for all small arithmetic circuits C that don’t compute the identically-zero polynomial, there exists an i such that C (xi)̸ = 0. What makes the derandomization “black-box” is that the choice of x1, . . . , xℓdoesn’t depend on C.
1997. u1997 While not hard to prove, Theorems 98 and 99 are conceptually interesting: they show that the entire hardness of 3Sat and of the permanent can be “concentrated” into a small number
1998. u1998 My difficulty is that this sort of “explicit obstruction” to computing 3Sat or the of instances.
1999. u1999 permanent, seems barely related to the sorts of explicit obstructions that GCT is seeking.
2000. u2000 The obstructions of Theorems 98 and 99 aren’t representation-theoretic; they’re simply lists of hard instances.
2001. u2001 Furthermore, a list like φ1, . . . , φℓor A1, . . . , Aℓis not an easy-to-verify witness that 3Sat or the permanent is hard, because we’d still need to check that the list worked against all of the exponentially many nO(1)-sized circuits.
2002. u2002 Having such a list reduces a two-quantifier (ΠP 2
2003. u2003 2 ) problem to a one-quantifier (NP) problem, but it still doesn’t put the problem in P—and we have no result saying that if, for example, the permanent is hard, then there must be obstructions that can be verified in nO(1) time.
2004. u2004 Perhaps the best we can say is that, if we proved the permanent was hard, then we’d immediately get, for every n, an “obstruction” that could be both found and verified in 0 time ( steps! But for all we know, the complexity of finding provable obstructions could
2005. u2005 to 0 as our understanding improved, without ever passing through nO(1). jump from exp
2006. u2006 Thus, I find, GCT’s suggestion to look for faster obstruction-finding (or obstruction-recognizing) algorithms is a useful guide, a heuristic, a way to organize our thoughts about how we’re going to find, say, an irrep ρ whose multiplicity blocks the embedding of the permanent into the determinant. But this is not the only path that should be considered.
2007. u2007 7
2008. u2008 Conclusions Some will say that this survey’s very length, the bewildering zoo of approaches and variations and
2009. u2009 ?= NP results and barriers that it covered, is a sign that no one has any real clue about the P problem—or at least, that I don’t.
2010. u2010 Among those who think that, perhaps someone will write a shorter survey that points unambiguously to the right way forward!
2011. u2011 But it’s also possible that this business, of proving brute-force search unavoidable, seems complicated because it is complicated.
2012. u2012 I confess to limited sympathy for the idea that someone will just set aside everything that’s already known, think hard about the structural properties of the sets of languages accepted by deterministic and nondeterministic polynomial-time Turing machines, and find a property that holds for one set but not the other, thereby proving P̸ = NP. For I keep coming back to the question: if a hands-off, aprioristic approach sufficed for P̸ = NP, then why did it apparently not suffice for all the weaker separations that we’ve surveyed here?
2013. u2013 At the same time, I hope our tour of the progress in lower bounds has made the case that there’s no reason (yet!) to elevate P ?= NP to some plane of metaphysical unanswerability, or assume it to be independent of the axioms of set theory, or anything like that.
2014. u2014 The experience of complexity theory, including the superpolynomial lower bounds that people did prove after struggle and effort, is consistent with P ?= NP being “merely a math problem”—albeit, a math problem that happens to be well beyond the current abilities of civilization, much like the solvability of the quintic in the 1500s, or Fermat’s Last Theorem in the 1700s. When we’re faced with such a problem, a natural response is to want to deepen our understanding of the entire subject (in this case, algorithms and computation) surrounding the problem—not merely because that’s a prerequisite to someday capturing the famous beast, but because regardless, the new knowledge gained along the way will hopefully find uses elsewhere.
2015. u2015 In our case, modern cryptography, quantum computing, and parts of machine learning could all be seen as flowers that bloomed in the garden of P ?= NP.
2016. u2016 Obviously I don’t know how P̸ = NP will ultimately be proved—if I did, this would be a different survey!
2017. u2017 It seems plausible that a successful approach might yield the stronger result NP̸ ⊂P/poly (i.e., that it wouldn’t take advantage of uniformity); that it might start by proving Valiant’s Conjecture (i.e., that the algebraic case would precede the Boolean one); and that it might draw on many areas of mathematics, but none of these things are even close to certain. Half-jokingly, I once remarked that those who still think seriously about P ?= NP now seem to be divided into “Team Yellow Books” (those hoping to use algebraic geometry, representation theory, and other sophisticated mathematics) and “Team Caffeinated Alien Reductions” (those hoping to use elementary yet convoluted and alien chains of logic, as in Williams’s proof of NEXP̸ ⊂ACC)— and that as far as I can tell, the two teams seem right now to be about evenly matched.
2018. u2018 It also seems plausible that a proof of P̸ = NP would be a beginning rather than an end—i.e., that the proof would require new insights about computation that then could be applied to many other problems besides P ?= NP. But given how far we are from the proof, it’s obviously premature to speculate about the fallout.
2019. u2019 The one prediction I feel confident in making is that the idea of “ironic complexity theory”—i.e., of a profound duality between upper and lower bounds, where the way to prove that there’s no fast algorithm for problem A is to discover fast algorithms for problems B, C, and D—is here to stay.
2020. u2020 As we saw, ironic complexity theory is at the core of Mulmuley’s view, but it’s also at the core of Williams’s proof of NEXP̸ ⊂ACC, which in other respects is about as far from GCT as possible.
2021. u2021 The natural proofs barrier also provides a contrapositive version of ironic complexity, showing how the nonexistence of efficient algorithms is often what prevents us from proving lower bounds. If 3Sat is ever placed outside P, it seems like a good bet that the proof will place many other problems inside P—or at any rate, in smaller complexity classes than was previously known.
2022. u2022 So, if we take an optimistic attitude (optimistic about proving intractability!), then which breakthroughs should we seek next?
2023. u2023 There are hundreds of possible answers to that question—we’ve already encountered some in this survey—but if I had to highlight a few:
2024. u2024 Prove lower bounds against nonuniform TC0—for example, by finding a better-than-brute-
2025. u2025 force deterministic algorithm to estimate the probability that a neural network accepts a
2026. u2026 Prove a lower bound better than n⌊d/2⌋on the rank of an explicit d-dimensional tensor, or
2027. u2027 construct one of the many other algebraic or combinatorial objects—rigid matrices, elusive
2028. u2028 functions, etc.—that are known to imply new circuit lower bounds (see Section 6).
2029. u2029 Advance proof complexity to the point where we could, for example, prove a superpolynomial
2030. u2030 lower bound on the number of steps needed to convert some n-input, polynomial-size Boolean
2031. u2031 circuit C into some equivalent circuit C′, via moves that each swap out a size-O (1) subcircuit
2032. u2032 for a different size-O (1) subcircuit with identical input/output behavior.106 105In 1999, Allender [20] showed that the permanent, and various other natural #P-complete problems, can’t be solved by LOGTIME-uniform TC0 circuits: in other words, constant-depth threshold circuits for which there’s an O (log n)-time algorithm to output the ith bit of their description, for any i. Indeed, these problems can’t be solved by LOGTIME-uniform TC0 circuits of size f (n), where f is any function that can yield an exponential when iterated a constant number of times. The proof uses a hierarchy theorem; it would be interesting to know whether it relativizes. 106Examples are deleting two successive NOT gates, or applying de Morgan’s laws. By the completeness of Boolean
2033. u2033 Prove a superlinear or superpolynomial lower bound on the number of 2-qubit quantum gates
2034. u2034 needed to implement some explicit n-qubit unitary transformation U, to within exponential
2035. u2035 (Remarkably, as far as anyone knows today, one could succeed at this without
2036. u2036 needing to prove any new classical circuit lower bound.
2037. u2037 On the other hand, it’s plausible
2038. u2038 that one would need to overcome a unitary version of the natural proofs barrier.)
2039. u2039 Clarify whether ACC has a natural proofs barrier, and prove ACC lower bounds for problems
2040. u2040 Clarify whether there’s an arithmetic natural proofs barrier, or something else preventing us
2041. u2041 from crossing the “chasm at depth three” (see Sections 6.5.2 and 6.5.3).
2042. u2042 Prove any lower bound on D (n), the determinantal complexity of the n×n permanent, better
2043. u2043 than Mignon and Ressayre’s n2/2 (Theorem 74).
2044. u2044 Derandomize polynomial identity testing—or failing that, prove some derandomization the-
2045. u2045 orem that implies a strong new circuit lower bound.
2046. u2046 Discover a new class of polynomial-time algorithms, especially but not only for computing
2047. u2047 Prove any interesting new lower bound by finding a representation-theoretic obstruction to
2048. u2048 Pinpoint additional special properties of 3Sat or the permanent that could be used to
2049. u2049 evade the natural proofs barrier, besides the few that are already known, such as symmetry-
2050. u2050 characterization, self-reducibility, and the ability to simulate machines in a hierarchy theorem.
2051. u2051 Perhaps my favorite challenge: find new, semantically-interesting ways to “hobble” the classes
2052. u2052 of polynomial-time algorithms and polynomial-size circuits, besides the ways that have al-
2053. u2053 ready been studied, such as restricted memory, restricted circuit depth, monotone gates only,
2054. u2054 arithmetic operations only, and restricted families of algorithms (such as DPLL and certain
2055. u2055 linear and semidefinite programming relaxations). Any such restriction that one discovers is
2056. u2056 effectively a new slope that one can try to ascend up the P̸ = NP mountain.
2057. u2057 Going even further on a limb, I’ve long wondered whether massive computer search could give any insight into complexity lower bound questions, beyond the relatively minor ways it’s been used for this purpose already (see, e.g., [275, 272]). For example, could we feasibly discover the smallest arithmetic circuits to compute the permanents of 4 × 4 and 5 × 5 and 6 × 6 matrices?
2058. u2058 And if we did, would examination of those circuits yield any clues about what to prove for general n?
2059. u2059 The conventional wisdom has always been “no” to both questions. For firstly, as far as anyone knows today, the computational complexity of such a search will grow not merely exponentially but doubly exponentially with n (assuming the optimal circuits that we’re trying to find grow exponentially themselves), and examination of the constants suggests that n = 4 might already be out of reach. And secondly, even if we knew the optimal circuits, they’d tell us nothing about the existence or algebra, one can give local transformation rules that suffice to convert any n-input Boolean circuit into any equivalent circuit using at most exp (n) moves.
2060. u2060 From talking to experts, this problem seems closely related to the problem of proving superpolynomial lower bounds for so-called Frege proofs, but is possibly easier.
2061. u2061 nonexistence of clever algorithms that start to win only at much larger values of n. After all, many of the theoretically efficient algorithms that we know today only overtake the na¨ıve algorithms for n’s in the thousands or millions.107
2062. u2062 These arguments have force. Even so, I think we should be open to the possibility that someday, advances in raw computer power, and especially theoretical advances that decrease the effective size of the search space, might change the calculus and open up the field of “experimental complexity theory.” One way that could happen would be if breakthroughs in GCT, or some other approach, brought the quest for “explicit obstructions” (say, to the 1000 × 1000 permanent having size-1020 circuits) within the range of computer search, if still not within the range of the human mind.
2063. u2063 Or of course, the next great advance might come from some direction that I didn’t even mention here, whether because no one grasps its importance yet or simply because I don’t. But regardless of what happens, the best fate this survey could possibly enjoy would be to contribute, in some tiny way, to making itself obsolete.
2064. u2064 8
2065. u2065 Acknowledgments I thank Andy Drucker, Ron Fagin, Harvey Friedman, William Gasarch, Bruno Grenet, Daniel Grier, Josh Grochow, Christian Ikenmeyer, Adam Klivans, Pascal Koiran, Greg Kuperberg, JM Landsberg, Ashley Montanaro, Mateus Oliveira, Bruce Smith, David Speyer, Noah Stephens-Davidowitz, Iddo Tzameret, Leslie Valiant, Avi Wigderson, Ryan Williams, and Jon Yard for helpful observations and for answering questions; as well as Daniel Seita for help with Figure 9. I especially thank Michail Rassias for his exponential patience with my delays completing this article.