-
Notifications
You must be signed in to change notification settings - Fork 0
/
search.xml
8880 lines (8880 loc) · 412 KB
/
search.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>Algorithm Design 14</title>
<url>/2023/12/29/Algorithm-Design-14/</url>
<content><![CDATA[<h2 id="chapter-10-local-search">Chapter 10 Local Search</h2>
<h3 id="game-theory-local-search">10.2 Game Theory & Local
Search</h3>
<ol type="1">
<li><p>Shapley Network Design Game</p>
<ol type="1">
<li><p>Description</p>
<ul>
<li><p>Given a directed graph <span
class="math inline">\(G(V,E)\)</span>, each edge <span
class="math inline">\(e\in E\)</span> having cost <span
class="math inline">\(c_e\)</span></p></li>
<li><p><span class="math inline">\(k\)</span> players, each having
source and target <span class="math inline">\((s_i,t_i)\)</span>. For
simplicity, let <span class="math inline">\(s_i=s\)</span>.</p>
<p>Player <span class="math inline">\(i\)</span> wants to go from <span
class="math inline">\(s_i\)</span> to <span
class="math inline">\(t_i\)</span>.</p></li>
<li><p>Set strategy for player <span class="math inline">\(i\)</span>:
<span class="math inline">\(\{\text{paths }s_i\to t_i\}\)</span></p>
<p>Strategy profile: <span
class="math inline">\(P=\{P_1,\cdots,P_k\}\)</span>, where <span
class="math inline">\(P_i\)</span> denotes a path <span
class="math inline">\(s_i\to t_i\)</span></p></li>
<li><p>Cost of player <span class="math inline">\(i\)</span>: <span
class="math display">\[
C_i(P)=\sum_{e\in P_i}\frac{c_e}{|\{j\in [k]:e\in P_j\}|}
\]</span></p></li>
<li><p>Each player is selfish, and wants to minimize its own
cost</p></li>
</ul></li>
<li><p>Nash Equilibrium (NE)</p>
<ol type="1">
<li><p>Def: We say a strategy profile <span
class="math inline">\(P=(P_1,\cdots,P_k)\)</span> is a Nash Equilibrium,
if <span class="math inline">\(\forall i\)</span>, <span
class="math inline">\(\forall P_i'\)</span> <span
class="math display">\[
C_i(P_1,\cdots,P_{i-1},P_i,P_{i+1},\cdots,P_n)\le
C_i(P_1,\cdots,P_{i-1},P_i',P_{i+1},P_n)
\]</span> Similar to "local minimum", any single player wants to change
its strategy.</p>
<p>But some players can collaborate to make their costs
decrease.</p></li>
<li><p>Social Optimal</p>
<p>Def: Minimum Steiner Tree connecting <span
class="math inline">\(s,t_1,\cdots,t_k\)</span></p>
<ul>
<li><p>Steiner Tree: Given a set <span class="math inline">\(S\subseteq
V\)</span>, find a connected graph <span
class="math inline">\((V',E')\)</span> that <span
class="math inline">\(S\subseteq V'\subseteq V\)</span>, and
minimize the sum of cost of all edges in <span
class="math inline">\(E'\)</span>.</p>
<p>Pollard Conjecture: <span class="math inline">\(\frac{\texttt{M
Steriner T}}{\texttt{M Spanning T}}\ge \frac{\sqrt
3}{2}\)</span></p></li>
</ul></li>
<li><p>THM: There is a game where the social cost of the unique NE is
<span class="math inline">\(\Theta(\log k)\)</span> times the social
optimum.</p>
<ul>
<li><p>Remark: this THM indicates that NE can be extremely bad.</p></li>
<li><p>Example:</p>
<p><span class="math inline">\(V=s\cup
T\cup\{t_1,\cdots,t_k\}\)</span></p>
<p><span class="math inline">\(E=\{(s,t_i,\frac{1}{i}):i\in
[k]\}\cup\{(T,t_i,0):i\in [k]\}\cup(s,T,1+\epsilon)\)</span></p>
<p>Social Optimal: All players use <span class="math inline">\(s\to T\to
t_i\)</span>, with <span
class="math inline">\(C_i=\frac{1+\epsilon}{k}\)</span>. Total Cost =
<span class="math inline">\(1+\epsilon\)</span>.</p>
<p>NE: the above state is not NE, finally NE would be player <span
class="math inline">\(i\)</span> use <span class="math inline">\(s\to
t_i\)</span>, with <span class="math inline">\(C_i=\frac{1}{i}\)</span>.
Total Cost = <span class="math inline">\(H_k\)</span>.</p></li>
</ul></li>
<li><p>NE Properties</p>
<ol type="1">
<li><p>Existence: NE can be achieved by best response dynamics
(BRP).</p>
<p>If BRP terminates (no cycle), it must be a NE.</p></li>
<li><p>Price of Anarchy(无政府主义): <span
class="math inline">\(PoA=\frac{Cost(\texttt{worst
NE})}{cost(\texttt{Social Optimal})}\)</span></p>
<p>Price of Stability: <span
class="math inline">\(PoS=\frac{Cost(\texttt{best
NE})}{cost(\texttt{Social Optimal})}\)</span></p></li>
</ol></li>
<li><p>THM: <span class="math inline">\(PoS\le \mathcal O(\log
k)\)</span></p>
<p>Proof: [potential function method]</p>
<p>Let <span class="math inline">\(H(k)=\sum_{i=1}^k
\frac{1}{i}\)</span>, and <span class="math inline">\(X_e=|\{i\in
[k]:e\in P_i\}|\)</span> <span class="math display">\[
\Phi(P):=\sum_{e\in E}c_e H(X_e)
\]</span></p>
<ul>
<li><p>Key Lemma: Suppose player <span class="math inline">\(i\)</span>
wants to update from <span class="math inline">\(P_i\)</span> to <span
class="math inline">\(P_i'\)</span>, then <span
class="math display">\[
C_i(P_i,P\backslash P_i)-C_i(P_i',P\backslash
P_i)=\Phi(P_i,P\backslash P_i)-\Phi(P_i',P\backslash P_i)
\]</span> Proof: <span class="math display">\[
\begin{aligned}
&\quad C_i(P_i,P\backslash P_i)-C_i(P_i',P\backslash P_i)\\
&=\sum_{e\in P_i}\frac{c_e}{X_e}-\sum_{e\in
P_i'}\frac{c_e}{X_e'}\\
&=\sum_{e\in P_i\backslash P_i'}\frac{c_e}{X_e}-\sum_{e\in
P_i'\backslash P_i}\frac{c_e}{X_e+1}\\
&=\left(\sum_{e\in P_i\backslash P_i'}c_eH(X_e)-\sum_{e\in
P_i\backslash P_i'}c_eH(X_e-1)\right)-\left(\sum_{e\in
P_i'\backslash P_i}c_eH(X_e+1)-\sum_{e\in P_i'\backslash
P_i}c_eH(X_e)\right)\\
&=\left(\sum_{e\in P_i\backslash P_i'}c_eH(X_e)-\sum_{e\in
P_i\backslash P_i'}c_eH(X_e')\right)-\left(\sum_{e\in
P_i'\backslash P_i}c_eH(X_e')-\sum_{e\in P_i'\backslash
P_i}c_eH(X_e)\right)\\
&=\left(\sum_{e\in P_i\backslash P_i'}c_eH(X_e)+\sum_{e\in
P_i'\backslash P_i}c_eH(X_e)\right)-\left(\sum_{e\in
P_i'\backslash P_i}c_eH(X_e')+\sum_{e\in P_i\backslash
P_i'}c_eH(X_e')\right)\\
&=\Phi(P_i,P\backslash P_i)-\Phi(P_i',P\backslash P_i)
\end{aligned}
\]</span> <span class="math inline">\(\square\)</span></p></li>
<li><p>Consider BRD, <span class="math inline">\(\Phi\)</span> decreases
monotonically, and BRD is finite, so BRD will terminate and lead to
NE.</p>
<p>Let <span class="math inline">\(P^*\)</span> denote the social
optimal potential, so, <span class="math display">\[
H_k\cdot C(P^*)\ge \Phi(P^*)>\Phi(P_{NE})\ge C(P_{NE})
\]</span></p></li>
</ul></li>
<li><p>Undirected graph case</p>
<p>Worst case Example not holds.</p>
<p>Current best lower bound: <span
class="math inline">\(PoA\ge1.8\)</span></p>
<p>Current best upper bound:</p>
<ul>
<li><span class="math inline">\(PoS\le \mathcal O(\log\log n)\)</span>
for broadcast game (each vertex has <span
class="math inline">\(t_i\)</span>, and <span
class="math inline">\(s_i=s\)</span>) [Fiat et.al.]</li>
<li><span class="math inline">\(PoS\le \mathcal O(\log\log\log
n)\)</span> for broadcast game [Lee, Ligett]</li>
<li><span class="math inline">\(PoS\le \mathcal O(1)\)</span> for
broadcast game [Bilo et.al. 20]</li>
<li><span class="math inline">\(PoS\le \mathcal O(\frac{\log n}{\log
\log n})\)</span> for multicast game</li>
</ul></li>
</ol></li>
</ol></li>
</ol>
<h2 id="chapter-11-linear-programming">Chapter 11 Linear
Programming</h2>
<h3 id="totally-unimodular-matrix">11.1 Totally Unimodular Matrix</h3>
<ol type="1">
<li><p>Definitions</p>
<ol type="1">
<li>[ <strong><em>Totally Unimodular Matrix,TUM</em></strong> ] Matrix
<span class="math inline">\(A\)</span> is TUM if the determinant of
every square submatrix belongs to <span class="math inline">\(\{0,\pm
1\}\)</span></li>
<li>[ <strong><em>Integral Polytope/Polyhedron</em></strong> ] A
polytope <span class="math inline">\(P\)</span> is integral if every
vertex of <span class="math inline">\(P\)</span> is an integral
vector</li>
</ol></li>
<li><p>Prop: If <span class="math inline">\(A\)</span> is TUM, for every
invertible square submatrix <span class="math inline">\(U\)</span> (i.e.
<span class="math inline">\(\det(U)=\pm 1\)</span>) , <span
class="math inline">\(U^{-1}\)</span> is integral (every entry in <span
class="math inline">\(U^{-1}\)</span> is integral)</p>
<p>Proof: <span
class="math inline">\(U_{ij}^{-1}=\frac{\det(U^*_{ji})}{\det(U)}=\frac{0,\pm
1}{\pm 1}\)</span></p></li>
<li><p>THM [ Hoffman, Kruscal ]: <span class="math inline">\(A\)</span>
is TUM <span class="math inline">\(\iff\)</span> For any integral vector
<span class="math inline">\(\vec b\)</span>, <span
class="math inline">\(P=\{\vec x:A\vec x\le \vec b\}\)</span> is
integral</p>
<p>Proof: <span class="math inline">\(\Rightarrow\)</span></p>
<p>A vertex is the solution of a linear system <span
class="math inline">\(A'\vec x=\vec b'\)</span> ( <span
class="math inline">\(A'\)</span> consists of a subset of rows <span
class="math inline">\(I\)</span> of <span
class="math inline">\(A\)</span> and full rank, <span
class="math inline">\(\vec b'=\vec b_I\)</span> ).</p>
<p>Therefore, <span class="math inline">\(A'\)</span> is invertible,
and <span class="math inline">\(\vec x=A'^{-1}\vec b'\)</span>.
By Prop, <span class="math inline">\(A'^{-1}\)</span> is integral,
so <span class="math inline">\(\vec x\)</span> is integral.</p></li>
<li><p>THM [ Judging TUM ]: <span class="math inline">\(A\in \mathbb
R^{m\times n}\)</span> is TUM <span class="math inline">\(\iff\)</span>
<span class="math inline">\(\forall R\subseteq [m]\)</span>, <span
class="math inline">\(\exists R=R_1+R_2\)</span>, s.t. <span
class="math inline">\(\forall j\in[n]\)</span>, <span
class="math inline">\(\sum\limits_{i\in R_1}a_{i,j}-\sum\limits_{i\in
R_2}a_{i,j}\in \{0,\pm 1\}\)</span>.</p>
<p>Proof: OMITTED</p></li>
<li><p>Example 1: Bipartite Matching</p>
<p>Maximize <span class="math inline">\(\sum_{e} x_e\)</span> , s.t.
<span class="math inline">\(\forall v,\sum_{v\in e}x_e\le 1\)</span>,
<span class="math inline">\(\forall e, x_e\ge 0\)</span>.</p>
<p><span class="math inline">\(A\in \mathbb R^{(|V|+|E|)\times
|E|}\)</span>, <span class="math inline">\(A_{v,e}=\mathbf 1(v\in
e)\)</span>, <span class="math inline">\(A_{|V|+e_i,e_j}=\mathbf
1(i=j)\)</span></p></li>
<li><p>Example 2: Consecutive "1" Matrix</p>
<p><span class="math inline">\(A\in \mathbb R^{m\times n}\)</span>,
<span class="math inline">\(\forall i\in [m]\)</span>, <span
class="math inline">\(\exist 1\le l_i\le r_i\le n\)</span>, <span
class="math inline">\(A_{i,j}=\mathbf 1(j\in [l_i,r_i])\)</span></p>
<p>Remark: Interval Scheduling</p></li>
<li><p>Example 3: Network Matrix</p>
<p>Given a directed tree <span class="math inline">\(T(V,E)\)</span>,
<span class="math inline">\(|E|=m\)</span>, and a set of ordered pairs
of vertices <span class="math inline">\(P\subseteq V\times V\)</span>,
<span class="math inline">\(|P|=k\)</span>.</p>
<p>Network Matrix: <span class="math inline">\(M\in \mathbb R^{m\times
k}\)</span> , if <span class="math inline">\(e=(u,v)\)</span>, <span
class="math inline">\(\bar e:=(v,u)\)</span> <span
class="math display">\[
M_{e,(v_1,v_2)}=\begin{cases}1&e\in Path(v_1,v_2)\\
-1&\bar e\in Path(v_1,v_2)\\
0&e,\bar e\notin Path(v_1,v_2)\end{cases}
\]</span> THM [ Tutte ]: A network matrix is TUM</p>
<p>Remark:</p>
<ul>
<li><p>Reverse result almost correct, only <span
class="math inline">\(2\)</span> classes of matrices are TUM but not
network matrix</p>
<p>TUM=Network Matrix+ (2 classes of matrices & their
operation)</p></li>
</ul></li>
<li><p>Remark: TDI, Matroid <span class="math inline">\(\sim\)</span>
TUM</p></li>
</ol>
]]></content>
<categories>
<category>课程笔记</category>
<category>算法设计</category>
</categories>
<tags>
<tag>博弈论-纳什均衡</tag>
<tag>博弈论-纳什均衡-PoA</tag>
<tag>博弈论-纳什均衡-PoS</tag>
<tag>算法-斯坦纳树</tag>
<tag>博弈论-Shapley网络设计游戏</tag>
<tag>算法-势能分析</tag>
<tag>线性规划-TUM</tag>
<tag>线性规划-整数线性规划</tag>
<tag>线性规划-网络矩阵</tag>
</tags>
</entry>
<entry>
<title>Algorithm Design 13</title>
<url>/2023/12/29/Algorithm-Design-13/</url>
<content><![CDATA[<h2 id="chapter-9-randomized-algorithm">Chapter 9 Randomized
Algorithm</h2>
<h3 id="approximate-counting">9.4 Approximate Counting</h3>
<ol type="1">
<li><p>Strategy : Rejection Selection</p>
<p>Goal : estimate <span class="math inline">\(|G|\)</span></p>
<p>Method : take <span class="math inline">\(U\)</span> s.t. <span
class="math inline">\(G\subseteq U\)</span></p>
<ul>
<li>We can take uniform samples from <span
class="math inline">\(U\)</span> efficiently</li>
<li>There is a membership oracle <span class="math inline">\(\mathcal
O\)</span> for <span class="math inline">\(G\)</span> : <span
class="math inline">\(\forall x\in U,\mathcal O(x)=\mathbf 1(x\in
G)\)</span></li>
<li>We can easily compute/estimate <span
class="math inline">\(|U|\)</span></li>
</ul>
<p>Theorem : Let <span
class="math inline">\(\alpha=\frac{|G|}{|U|}\)</span>, so we need <span
class="math inline">\(N\ge
c\frac{1}{\epsilon^2\alpha}\log(\frac{1}{\delta})\)</span> samples, s.t.
<span class="math display">\[
\Pr\left\{\frac{\text{\# samples in }G}{N}\in (1\pm
\epsilon)|G|\right\}\ge 1-\delta
\]</span> Proof : Chernoff Bound</p></li>
<li><p>Counting #DNF solutions</p>
<ol type="1">
<li><p>Description</p>
<p>DNF : <span class="math inline">\(\lor_{i=1}^m C_i\)</span>, <span
class="math inline">\(C_i=a_{i,1}\land \cdots\land a_{i,l_i}\)</span> ,
<span class="math inline">\(a_{i,j}\in \{x_t,\bar x_t:1\le t\le
n\}\)</span></p>
<p>Goal : Estimate the number of satisfying assignments ( out of <span
class="math inline">\(2^n\)</span> possibilities )</p>
<p>#P Hard Problem</p></li>
<li><p>Karp-Luby-Mardars Method</p>
<p>If DNF has a clause of size <span
class="math inline">\(3\)</span>:</p>
<p><span class="math inline">\(U\)</span> : all <span
class="math inline">\(2^n\)</span> assignments, <span
class="math inline">\(G\)</span> : satisfying assignments</p>
<p><span class="math inline">\(|G|\ge \frac{|U|}{8}\)</span>, so <span
class="math inline">\(\alpha\ge \frac{1}{8}\)</span>, we only need <span
class="math inline">\(\mathcal O(\frac{1}{\epsilon^2}\log
\frac{1}{\delta})\)</span></p>
<p>For general case:</p>
<p>Considering a table, <span class="math inline">\(2^n\times
m\)</span>, <span class="math inline">\(t_{i,j}=*\)</span> iff
assignment <span class="math inline">\(i\)</span> can satisfy clause
<span class="math inline">\(j\)</span>. <span
class="math inline">\(t_{i,j}=\otimes\)</span> if clause <span
class="math inline">\(j\)</span> is the first clause satisfied by
assignment <span class="math inline">\(i\)</span></p>
<p><span class="math inline">\(U\)</span> : All <span
class="math inline">\(*\)</span> , <span
class="math inline">\(G\)</span> : All <span
class="math inline">\(\otimes\)</span></p>
<ol type="1">
<li><p>How to sample from <span class="math inline">\(U\)</span> :</p>
<p><span class="math inline">\(|U|=\sum_{i=1}^m 2^{n-l_i}\)</span> .</p>
<ul>
<li>Firstly sample column <span class="math inline">\(i\)</span> w.p.
<span class="math inline">\(\frac{2^{n-l_i}}{|U|}\)</span> ( this can be
implemented in <span class="math inline">\(\mathcal O(1)\)</span> , or
<span class="math inline">\(\mathcal O(m)\)</span> ?)</li>
<li>Then, from column <span class="math inline">\(i\)</span>, choose
<span class="math inline">\(*\)</span></li>
</ul></li>
<li><p>Check whether <span class="math inline">\(*\in G\)</span> :
easily in <span class="math inline">\(\mathcal O(m)\)</span> , check all
previous clauses</p></li>
<li><p><span class="math inline">\(\alpha=\frac{|G|}{|U|}\ge
\frac{1}{m}\)</span> : we have at least one <span
class="math inline">\(*\)</span> in one satisfying assignment</p></li>
</ol></li>
</ol></li>
</ol>
<h3 id="streaming-algorithms">9.5 Streaming Algorithms</h3>
<ol type="1">
<li><p>Definition</p>
<p>Input arrives in stream fashion, length of stream is <span
class="math inline">\(n\)</span>.</p>
<p>Goal: estimate some statistics of the stream.</p>
<p>Setting: Space $ n$ . Typical size : <span
class="math inline">\(\mathcal O(1),poly(\log n),\mathcal O(\sqrt
n)\)</span> ( at least sublinear )</p>
<p>Usually, we cannot give a precise solution due to space
constraint.</p>
<p><span class="math inline">\((\epsilon,\delta)\)</span>-solution:
<span class="math inline">\(\Pr\{\mathtt{SOL}\in (1\pm \epsilon)\texttt{
True Value}\}\ge 1-\delta\)</span>.</p></li>
<li><p>Counting the distinct elements ( <span
class="math inline">\(F_0\)</span>-estimation )</p>
<ol type="1">
<li><p>Description</p>
<p><span class="math inline">\(n\)</span> elements, each element <span
class="math inline">\(\in [m]\)</span> , <span
class="math inline">\(n,m\)</span> are large</p>
<p>Goal: output the number of distinct elements</p></li>
<li><p>Google : HyperLogLog</p></li>
<li><p>A simpler (idealized) algorithm with Hash function</p>
<ol type="1">
<li><p>deterministic hash function <span class="math inline">\(h:[m]\to
[0,1]\)</span></p></li>
<li><p>Take representative: maintain <span
class="math inline">\(k=\mathcal O(\frac{1}{\epsilon^2})\)</span>-th
smallest <span class="math inline">\(h\)</span> value ( suppose its
value is <span class="math inline">\(t\)</span> )</p>
<blockquote>
<p>Not taking the smallest value: variance is large</p>
</blockquote></li>
<li><p>Estimation: <span
class="math inline">\(\frac{t}{k}\)</span></p></li>
</ol></li>
<li><p>Can achieve <span
class="math inline">\((\epsilon,\delta)\)</span>-approximation (Proof:
Chernoff Bound)</p></li>
<li><p>Real Implementation: <span class="math inline">\(h:[m]\to
[m^2]\)</span>, only store smallest <span
class="math inline">\(k\)</span> values</p></li>
</ol></li>
</ol>
<h2 id="chapter-10-local-search">Chapter 10 Local Search</h2>
<h3 id="metropolis-algorithm">10.1 Metropolis Algorithm</h3>
<ol type="1">
<li><p>From statistical mechanics</p>
<p><span class="math inline">\(\Pr\{\text{a state with energy }E\}\sim
\exp(-\frac{E}{kT})\)</span></p>
<p><img
src="C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20231218110241961.png" /></p>
<p>(Purple: Small <span class="math inline">\(T\)</span>, Red: Large
<span class="math inline">\(T\)</span>)</p></li>
<li><p>Markov Chain</p>
<p>Directed weighted graph <span class="math inline">\(G\)</span>, <span
class="math inline">\(\forall u\in V,\sum_{(u,v)\in
E}p(u,v)=1\)</span></p>
<p>Transition matrix: <span
class="math inline">\(P_{i,j}=p(i,j)=P(j|i)\)</span></p>
<p><span class="math inline">\((P^k)_{i,j}\)</span>: probability that
start from <span class="math inline">\(i\)</span> and end at <span
class="math inline">\(j\)</span> with exactly <span
class="math inline">\(k\)</span> steps</p>
<p>Stationary distribution <span class="math inline">\(\bar x\)</span>:
<span class="math inline">\(P^T\bar x=\bar x\)</span></p>
<p>Finite space: must have stationary distribution</p></li>
<li><p>Count # perfect matching in bipartite graph</p>
<p><span class="math inline">\(\iff\)</span> Sampling problem <span
class="math inline">\(\iff\)</span> Markov Chain ( matching <span
class="math inline">\(\to\)</span> matching )</p>
<p>Design MC s.t. stationary distribution of MC is uniformly distributed
over all states</p>
<p>Starting from one state, after <span class="math inline">\(k\)</span>
steps we stop, consider the final state as a uniformly sampled
stationary distribution</p>
<p>Under certain condition (Ergodic), a random walk will converge to a
<strong>unique</strong> stationary distribution</p>
<p><span class="math inline">\(\forall
x_0,\lim\limits_{k\to\infty}(P^T)^k x_0=\bar x\)</span></p>
<p>Mix time: maximum ( consider all <span
class="math inline">\(x_0\)</span> ) time to reach <span
class="math inline">\(\bar x\)</span></p></li>
<li><p>Metropolis Algorithm</p>
<p><span class="math inline">\(\mathcal C\)</span>: set of all states,
<span class="math inline">\(S\)</span>: current state, <span
class="math inline">\(S'\)</span>: a uniformly chosen neighbor of
<span class="math inline">\(S\)</span>.</p>
<p>If <span class="math inline">\(E(S')<E(S)\)</span>, <span
class="math inline">\(S\gets S'\)</span>. Otherwise, <span
class="math inline">\(S\gets S'\)</span> with probability <span
class="math inline">\(\exp(-\frac{E(S')-E(S)}{kT})\)</span>.</p>
<p>Theorem: Let <span class="math inline">\(Z=\sum_{S\in \mathcal
C}\exp(-\frac{E(S)}{kT})\)</span>, then <span class="math display">\[
\Pr\{S\gets \mathcal C:S\in \text{stationary
distribution}\}=\frac{1}{Z}\exp(-\frac{E(S)}{kT})
\]</span> Application:</p>
<ul>
<li>Sampling: Bayesian Inference/ Graphical Models</li>
<li>Optimization: minimize <span class="math inline">\(f(x)\)</span>:
Let <span class="math inline">\(E(S)=f(S)\)</span> <span
class="math inline">\(\to\)</span> Simulated Annealing</li>
</ul></li>
<li><p>Simulated Annealing</p>
<p>Goal: minimize <span class="math inline">\(f\)</span></p>
<ul>
<li>High temperature: like uniform, Markov Chain converges quicker for
larger <span class="math inline">\(T\)</span> (since the graph has good
continuity)</li>
<li>Low temperature: many local minimum, hard to sample (converges
slower)</li>
</ul>
<p>Cooling scheduling <span class="math inline">\(T=T(i)\)</span>: <span
class="math inline">\(i=1,2,\cdots\)</span></p>
<ul>
<li>$x_i$ run Metropolis Algorithm for <span
class="math inline">\(T(i)\)</span> with initial state <span
class="math inline">\(x_{i-1}\)</span></li>
</ul></li>
<li><p>Mixing Time</p>
<p>Total variation distance: <span class="math inline">\(p,q\)</span>
are two distributions, <span class="math display">\[
d_{TV}(p,q)=\sum_{i}|p_i-q_i|=\left\|\vec p-\vec q\right\|_1
\]</span> Let <span class="math inline">\(p_x^k=(P^T)^kx\)</span>, <span
class="math inline">\(\pi\)</span> be the stationary distribution.
Define mixing time as <span class="math display">\[
\tau_x(\epsilon)=\min\{k:d_{TV}(p_x^k,\pi)\le \epsilon\}
\]</span> Mixing time <span class="math inline">\(\sim\)</span>
continuity</p>
<ul>
<li>MinCut very small <span class="math inline">\(\to\)</span> mixing
time large</li>
<li>Good continuity <span class="math inline">\(\to\)</span> mixing time
small</li>
</ul>
<p>Theorem: Let <span class="math inline">\(P\)</span> be the transition
matrix of Markov Chain, suppose <span class="math inline">\(P\)</span>
is symmetric. Suppose <span class="math inline">\(\lambda_1\ge
\lambda_2\ge \cdots\ge \lambda_N\)</span> be eigenvalues of <span
class="math inline">\(P\)</span>, it is known that <span
class="math inline">\(\lambda_1=1\)</span> and <span
class="math inline">\(v_1=(\frac{1}{\sqrt n},\cdots,\frac{1}{\sqrt
n})\)</span>. Then let <span
class="math inline">\(\lambda_{\max}=\max\{|\lambda_2|,|\lambda_N|\}\)</span>,
so <span class="math display">\[
\tau(\epsilon)\le \mathcal O\left(\frac{\log n+\log
\frac{1}{\epsilon}}{1-\lambda_{\max}}\right)
\]</span> i.e. <span class="math inline">\(\lambda_2\to 1\)</span>,
mixing time <span class="math inline">\(\to\)</span> large</p></li>
</ol>
]]></content>
<categories>
<category>课程笔记</category>
<category>算法设计</category>
</categories>
<tags>
<tag>算法-随机算法</tag>
<tag>算法-随机算法-近似计数问题</tag>
<tag>算法-随机算法-采样与计数</tag>
<tag>算法-随机算法-近似计数问题-KLM方法</tag>
<tag>算法-流算法-不同元素个数</tag>
<tag>算法-局部搜索-Metropolis算法</tag>
<tag>机器学习-马尔科夫链</tag>
<tag>算法-局部搜索-模拟退火</tag>
</tags>
</entry>
<entry>
<title>Algorithm Design 12</title>
<url>/2023/12/29/Algorithm-Design-12/</url>
<content><![CDATA[<h2 id="chapter-9-randomized-algorithm">Chapter 9 Randomized
Algorithm</h2>
<h3 id="hashing">9.3 Hashing</h3>
<ol type="1">
<li><p>universal hashing function</p>
<p>Intuition : want the function looks "random"</p>
<ol type="1">
<li><p>Def [ <strong><em>universal hashing function</em></strong> ] A
family <span class="math inline">\(\mathcal H\)</span> of functions
<span class="math inline">\(h:U\to\{0,\cdots,n-1\}\)</span> is
<strong><em>universal</em></strong> if <span
class="math inline">\(\forall u,v\in U,u\neq v\)</span> , <span
class="math display">\[
\Pr\{h\gets \mathcal H:h(u)=h(v)\}\le \frac{1}{n}
\]</span></p></li>
<li><p>Design : Let <span class="math inline">\(p\approx n\)</span> be a
prime number ( <span class="math inline">\(n\)</span> is the size of
hash table , for simplicity let <span class="math inline">\(p=n\)</span>
)</p>
<p>For all <span class="math inline">\(x\in U\)</span> , write <span
class="math inline">\(x\)</span> in base-<span
class="math inline">\(p\)</span> : <span
class="math inline">\(x=\sum_{i=0}^{r-1} x_ip^i\)</span> , where <span
class="math inline">\(x_i\in \{0,\cdots,p-1\}\)</span> . <span
class="math display">\[
\mathcal H:=\left\{h_{\vec
a}(x)=\left(\sum_{i=0}^{r-1}a_ix_i\right)\bmod p:\vec
a=(a_0,\cdots,a_{r-1}),a_i\in \{0,\cdots,p-1\}\right\}
\]</span> Goal : prove <span class="math inline">\(\forall x,y\in
U,x\neq y\)</span> , <span class="math display">\[
\Pr\{a_i\gets \{0,\cdots,p-1\}:h_{\vec a}(x)=h_{\vec a}(y)\}\le
\frac{1}{n}
\]</span> Proof :</p>
<p>Suppose <span class="math inline">\(x=\sum_{i=0}^{r-1}x_ip^i\)</span>
, <span class="math inline">\(y=\sum_{i=0}^{r-1}y_ip^i\)</span> , so
there exists <span class="math inline">\(j\)</span> such that <span
class="math inline">\(x_j\neq y_j\)</span> . <span
class="math display">\[
\begin{aligned}
&\quad \Pr\left\{h_{\vec a}(x)=h_{\vec a}(y)\right\}\\
&=\Pr\left\{\sum_{i=0}^{r-1}a_ix_i\equiv \sum_{i=0}^{r-1}a_iy_i\pmod
p\right\}\\
&=\Pr\left\{a_j(x_j-y_j)\equiv \sum_{i=0,i\neq
j}^{r-1}a_i(y_i-x_i)\pmod p\right\}\\
&=\frac{1}{p}
\end{aligned}
\]</span> This is because <span class="math inline">\(a_j\gets
\{0,\cdots,p-1\}\)</span> , and for all possible <span
class="math inline">\(\sum_{i=0,i\neq j}^{r-1}a_i(y_i-x_i)\)</span> ,
there exists exactly one <span class="math inline">\(a_j\)</span> that
<span class="math inline">\(a_j(x_j-y_j)\equiv \sum_{i=0,i\neq
j}^{r-1}a_i(y_i-x_i)\pmod p\)</span></p></li>
</ol></li>
<li><p>Perfect Hashing</p>
<ol type="1">
<li><p>Def : static dictionary with no collision , <span
class="math inline">\(|U|=n\)</span> , space <span
class="math inline">\(\mathcal O(n)\)</span> , query time <span
class="math inline">\(\mathcal O(1)\)</span> ( with <span
class="math inline">\(h(x)\)</span> oracle , performing <span
class="math inline">\(h(x)\)</span> costs <span
class="math inline">\(\mathcal O(1)\)</span> time )</p>
<p>static : only consider lookup operation , no insertion/deletion
.</p></li>
<li><p>FKS Perfect Hashing [1984]</p>
<p>Intuition : <span class="math inline">\(2\)</span>-level data
structure</p>
<ul>
<li>Firstly sample <span class="math inline">\(h\gets\mathcal H\)</span>
, get a hash table , space <span class="math inline">\(\mathcal
O(n)\)</span></li>
<li>Secondly , if index <span class="math inline">\(i\)</span>
encounters collision (suppose the collision set is <span
class="math inline">\(B_i=\{u\in U:h(u)=i\}\)</span> ), then sample
<span class="math inline">\(h_i\gets \mathcal H'\)</span> and get a
second-level hash table , space <span class="math inline">\(\mathcal
O(|B_i|^2)\)</span> . If collision still happens , then resample <span
class="math inline">\(h_i\)</span> , until no collision happens .</li>
</ul></li>
<li><p>Collision Analysis</p>
<p>Claim : <span class="math inline">\(\mathcal H=\{h\}\)</span>
universal , <span class="math inline">\(h:[n]\to [m]\)</span> . If <span
class="math inline">\(m=\mathcal O(n^2)\)</span> , then with probability
<span class="math inline">\(\ge 0.9\)</span> there is no collision .</p>
<p>Proof : <span class="math inline">\(U=\{x_1,\cdots,x_n\}\)</span> ,
define random variable <span
class="math inline">\(X_{i,j}=\begin{cases}1&h(x_i)=h(x_j)\\0&otherwise\end{cases}\)</span>
, define <span class="math inline">\(X=\sum_{1\le i<j\le n}
X_{i,j}\)</span> <span class="math display">\[
E[X]=\sum_{i<j}E[X_{i,j}]=\sum_{i<j}\Pr\{h(x_i)=h(x_j)\}\le
\frac{\binom{n}{2}}{M}
\]</span> By Markov's Inequality , <span class="math display">\[
\Pr\{X\ge 1\}\le E[X]\le \frac{n(n-1)}{2M}\le \frac{n^2}{2M}
\]</span> Therefore , let <span class="math inline">\(M=5n^2\)</span> ,
so <span class="math inline">\(\Pr\{X\ge 1\}\le \frac{1}{10}\)</span> .
<span class="math inline">\(\square\)</span></p></li>
<li><p>Space Analysis</p>
<p>Claim : <span class="math inline">\(E[\sum |B_i|^2]=\mathcal
O(n)\)</span></p>
<p>Proof : Let <span class="math inline">\(Y_i=|B_i|\)</span> , <span
class="math inline">\(Y=\sum |B_i|^2\)</span> , so in the first level ,
<span class="math display">\[
X=\sum \binom{Y_i}{2}=\frac{1}{2}\left(\sum Y_i^2-\sum
Y_i\right)=\frac{Y-n}{2}
\]</span> Similarly , <span class="math inline">\(E[X]\le
\frac{\binom{n}{2}}{n}=\frac{n-1}{2}\)</span> , so <span
class="math inline">\(E[Y]=E[2X+n]\le 2n-1=\mathcal O(n)\)</span>
.</p></li>
</ol></li>
<li><p>Balls-Bins Game</p>
<ol type="1">
<li><p><span class="math inline">\(n\)</span> bins , <span
class="math inline">\(n\)</span> balls , <span
class="math inline">\(E[\text{max bins}]=\Theta(\frac{\log n}{\log\log
n})\)</span></p>
<p><span class="math inline">\(X_i=\#\text{ balls in bin }i\)</span> ,
<span class="math inline">\(Y_{i,j}=\begin{cases}1&\text{ball }j\to
\text{bin }i\\0&otherwise\end{cases}\)</span> , so <span
class="math inline">\(X_i=\sum Y_{i,j}\)</span></p>
<p>Let <span class="math inline">\(c=\Theta(\frac{\log n}{\log\log
n})\)</span> , so <span class="math inline">\(c=1+\delta\)</span> ,
<span class="math inline">\(E[X_i]=1\)</span> , so <span
class="math display">\[
\Pr\{X_i\ge c\}\le \left(\frac{e^{c-1}}{c^c}\right)\le \frac{1}{n^2}
\]</span></p></li>
<li><p>Coupon Collector Problem</p>
<p><span class="math inline">\(n\ln n\)</span> balls , <span
class="math inline">\(n\)</span> bins , w.h.p. every bin is
non-empty</p></li>
<li><p>Load Balancing Problem</p>
<p><span class="math inline">\(kn\log n\)</span> balls , <span
class="math inline">\(n\)</span> bins , load of every bin <span
class="math inline">\(\approx (k-1,k+1)\log n\)</span> ( <span
class="math inline">\(k\)</span> not too small )</p>
<p>Application : load balancing in network</p></li>
<li><p>Power of two choices</p>
<p><span class="math inline">\(n\)</span> bins , <span
class="math inline">\(n\)</span> balls</p>
<p>For every ball , choose <span class="math inline">\(2\)</span> random
bins , and place the ball in the bin with smaller load .</p>
<p><span class="math inline">\(E[\text{max bins}]=\Theta(\log \log
n)\)</span></p></li>
</ol></li>
<li><p>(*) Cuckoo Hashing</p>
<ol type="1">
<li><p>worst case : <span class="math inline">\(\mathcal O(1)\)</span>
lookup , dynamic dictionary</p></li>
<li><p>Maintain <span class="math inline">\(2\)</span> hash tables ,
<span class="math inline">\(h_1,h_2\gets \mathcal H\)</span></p>
<p><code>Insert(x)</code></p>
<ol type="1">
<li><p>If <span class="math inline">\(h_1(x)\)</span> in <span
class="math inline">\(T_1\)</span> is empty , insert <span
class="math inline">\(x\)</span> into <span
class="math inline">\(T_1\)</span></p></li>
<li><p>If <span class="math inline">\(T_1(h_1(x))\)</span> contains
<span class="math inline">\(x'\)</span> , then insert <span
class="math inline">\(x'\)</span> into <span
class="math inline">\(T_2\)</span> by <span
class="math inline">\(h_2(x')\)</span> , and insert <span
class="math inline">\(x\)</span> into <span
class="math inline">\(T_1\)</span> by <span
class="math inline">\(h_1(x)\)</span></p>
<p>Following some procedure , until an empty slot is found .</p></li>
<li><p>If #iteration $$ threshold <span class="math inline">\(t\)</span>
, rehash all elements</p></li>
</ol>
<p>If load <span class="math inline">\(\le 50\%\)</span> , <span
class="math inline">\(E[\texttt{insertion time}]=\mathcal
O(1)\)</span></p>
<p><code>Lookup(x)</code> : Obviously at most look up <span
class="math inline">\(2\)</span> times .</p>
<p><code>Delete(x)</code> : Use lookup then delete .</p></li>
</ol></li>
<li><p>Application : Closest Pair Problem</p>
<p>using D&C , <span class="math inline">\(\mathcal O(n\log
n)\)</span></p>
<p>using Hashing , <span class="math inline">\(\mathcal O(n)\)</span> in
expectation</p>
<ol type="1">
<li><p>Hash function computation modal :</p>
<ul>
<li>Insertion : <span class="math inline">\(\mathcal O(1)\)</span> in
expectation</li>
<li>Deletion : <span class="math inline">\(\mathcal O(1)\)</span> in
expectation</li>
<li>Lookup : <span class="math inline">\(\mathcal O(1)\)</span> in
expectation</li>
</ul></li>
<li><p>Algorithm :</p>
<ol type="1">
<li><p>Order points randomly <span
class="math inline">\(P_1,\cdots,P_n\)</span> , let <span
class="math inline">\(\delta=d(P_1,P_2)\)</span></p></li>
<li><p>When we process <span class="math inline">\(P_i\)</span> , we
will find whether <span class="math inline">\(\exists j<i\)</span>
s.t. <span class="math inline">\(d(P_j,P_i)<\delta\)</span> .</p>
<p>If YES , we start a new phase with new <span
class="math inline">\(\delta'=d(P_j,P_i)\)</span></p></li>
<li><p>In each phase , maintain a grid with edge length <span
class="math inline">\(\frac{\delta}{2}\)</span> , each cell contains
<span class="math inline">\(\le 1\)</span> points in <span
class="math inline">\(\{P_1,\cdots,P_{i-1}\}\)</span></p>
<p>maintain all nonempty cells in a dictionary ( <span
class="math inline">\(U\)</span> : all cells )</p>
<p>Suppose we are processing <span class="math inline">\(P_i\)</span> ,
we only need to look up <span class="math inline">\(25\)</span> cells in
the dictionary to find smaller distance .</p>
<p>Whenever we start a new phase , we reconstruct the whole dictionary
.</p></li>
</ol></li>
<li><p>Analysis :</p>
<p><span class="math inline">\(X_i=\begin{cases}1&P_i\text{ causes a
new phase}\\0&otherwise\end{cases}\)</span> , running time <span
class="math inline">\(T=\sum_{i=1}^n \left(X_i \mathcal O(i)+25\mathcal
O(1)\right)\)</span> .</p>
<p>Therefore , <span class="math inline">\(E[T]=\left(\sum_{i=1}^n
\mathcal O(i)\Pr\{X_i=1\}\right)+\mathcal O(n)\)</span> .</p>
<p>Claim : <span class="math inline">\(\Pr\{X_i=1\}\le
\frac{2}{i}\)</span> ( by the initial random permutation )</p>
<p>Proof : Suppose <span class="math inline">\((P_a,P_b)\)</span> has
the minimum distance among <span
class="math inline">\(P_1,\cdots,P_i\)</span> , so <span
class="math inline">\(\Pr\{X_i=1\}=\Pr\{a=i\lor b=i\}\le
\Pr\{a=i\}+\Pr\{b=i\}\)</span> .</p>
<p><span
class="math inline">\(\Pr\{a=i\}=\Pr\{b=i\}=\frac{1}{i}\)</span> , so
<span class="math inline">\(\Pr\{X_i=1\}\le \frac{2}{i}\)</span> . <span
class="math inline">\(\square\)</span></p></li>
</ol></li>
</ol>
<h3 id="approximate-counting">9.4 Approximate Counting</h3>
<ol type="1">
<li><p>Approximate Counting Problem</p>
<ol type="1">
<li><p>#P : counting problem <span
class="math inline">\(\leftrightarrow\)</span> NP</p></li>
<li><p>#P-complete</p>
<p><span class="math inline">\(L\in \texttt{NP-Complete}\to L\in
\texttt{\#P-Complete}\)</span></p>
<p><span class="math inline">\(L\in \texttt{\#P-Complete}\not\to L\in
\texttt{NP-Complete}\)</span> : e.g. counting perfect matching</p></li>
<li><p>FPRAS : Fully-Poly Randomized Approximation Scheme</p>
<p>An algorithm in <span
class="math inline">\(poly(n,\frac{1}{\epsilon})\)</span> , <span
class="math inline">\(\Pr\{(1-\epsilon)ANS\le SOL\le
(1+\epsilon)ANS\}\ge 0.99\)</span></p></li>
<li><p>Scheme : Rejection Sampling</p>
<p>Example:</p>
<p><img src="/images/posts/AD12_fig1.png" /></p>
<p>General : to estimate <span class="math inline">\(|G|\)</span> ,
construct <span class="math inline">\(U\)</span> s.t. <span
class="math inline">\(G\subseteq U\)</span></p>
<ul>
<li>Can uniformly sample from <span
class="math inline">\(U\)</span></li>
<li>Membership Oracle : <span class="math inline">\(\forall x\in
U\)</span> , decide whether <span class="math inline">\(x\in G\)</span>
or not</li>
<li>We know <span class="math inline">\(|U|\)</span></li>
</ul></li>
</ol></li>
</ol>
]]></content>
<categories>
<category>课程笔记</category>
<category>算法设计</category>
</categories>
<tags>
<tag>算法-随机算法</tag>
<tag>算法-随机算法-哈希</tag>
<tag>算法-随机算法-近似计数问题</tag>
<tag>算法-随机算法-采样与计数</tag>
<tag>算法-随机算法-哈希-通用哈希</tag>
<tag>算法-随机算法-哈希-FKS完美哈希</tag>
<tag>算法-随机算法-球盒模型</tag>
<tag>算法-随机算法-Power of 2 choices</tag>
<tag>算法-随机算法-哈希-Cuckoo哈希</tag>
<tag>算法-随机算法-哈希-平面最近点对</tag>
</tags>
</entry>
<entry>
<title>Algorithm Design 11</title>
<url>/2023/12/29/Algorithm-Design-11/</url>
<content><![CDATA[<h2 id="chapter-8-network-flow">Chapter 8 Network Flow</h2>
<h3 id="application">8.3 Application</h3>
<h3 id="project-selection-problem">8.3.7 Project Selection Problem</h3>
<ol type="1">
<li><p>Description</p>
<p>A set of projects , project <span class="math inline">\(i\)</span>
has profit <span class="math inline">\(p_i\)</span> ( <span
class="math inline">\(p_i\)</span> can be positive or negative)</p>
<p>A set of procedure constraints : given by DAG , <span
class="math inline">\(e=(i,j)\)</span> means <span
class="math inline">\(i\)</span> can be selected only if <span
class="math inline">\(j\)</span> is selected .</p>
<p>Goal : Select a subset of projects <span
class="math inline">\(A\)</span> , s.t. <span
class="math inline">\(\sum_{i\in A}p_i\)</span> is maximized .</p></li>
<li><p>Flow Construction</p>
<p>Min-Cut Model</p>
<ul>
<li><span class="math inline">\((i,j)\)</span> : give <span
class="math inline">\(\infty\)</span> capacity , then it cannot be a cut
. Then it cannot be <span class="math inline">\(i\)</span> in <span
class="math inline">\(s\)</span> part while <span
class="math inline">\(j\)</span> in <span
class="math inline">\(t\)</span> part .</li>
<li>Define in <span class="math inline">\(s\)</span> part : selected ,
in <span class="math inline">\(t\)</span> part : not selected .</li>
<li>If <span class="math inline">\(p_i\ge 0\)</span> , then have edge
<span class="math inline">\((s,i,p_i)\)</span> ; If <span
class="math inline">\(p_i<0\)</span> , then have edge <span
class="math inline">\((i,t,-p_i)\)</span> .</li>
<li>Answer is <span class="math inline">\(\sum_{p_i\ge
0}p_i-\mathtt{MinCut}\)</span></li>
</ul>
<p>Formal construction :</p>
<ul>
<li><p>Vertices : <span class="math inline">\(V'=V\cup
\{s,t\}\)</span></p></li>
<li><p>Edges :</p>
<ul>
<li><p><span class="math inline">\(\forall (i,j)\in E\)</span> , <span
class="math inline">\((i,j,\infty)\)</span></p></li>
<li><p><span class="math inline">\(\forall i\in V,p_i\ge 0\)</span> ,
<span class="math inline">\((s,i,p_i)\)</span></p></li>
<li><p><span class="math inline">\(\forall i\in V,p_i<0\)</span> ,
<span class="math inline">\((i,t,-p_i)\)</span></p></li>
</ul></li>
<li><p>Find a Min-Cut in <span class="math inline">\(G'\)</span> :
<span class="math inline">\((A,V'-A)\)</span></p>
<p>The minimum value of cut <span
class="math inline">\((A,V'-A)\)</span> is <span
class="math display">\[
\begin{aligned}
&\quad \sum_{p_i\ge 0,i\notin A}p_i+\sum_{p_i<0,i\in A}(-p_i)\\
&=\sum_{p_i\ge 0}p_i-\left(\sum_{p_i\ge 0,i\in
A}p_i+\sum_{p_i<0,i\in A}p_i\right)
\end{aligned}
\]</span> Which is equivalent to maximize <span class="math display">\[
\sum_{p_i\ge 0,i\in A}p_i+\sum_{p_i<0,i\in A}p_i=\sum_{i\in A}p_i
\]</span></p></li>
</ul></li>
</ol>
<h4 id="densest-subgraph-problem">8.3.8 Densest Subgraph Problem</h4>
<ol type="1">
<li><p>Description</p>
<p>Given undirected graph <span class="math inline">\(G=(V,E)\)</span> .
Density of induced subgraph <span class="math inline">\(G[S]\)</span> ,
<span class="math inline">\(S\subseteq V\)</span> is defined as <span
class="math inline">\(\frac{|E\cap G[S]|}{|S|}\)</span></p>
<p>Goal : find a subgraph of maximum density .</p></li>
<li><p>More generalized (but easier problem) : Baseball Elimination</p>
<p>Input :</p>
<ul>
<li>A set <span class="math inline">\(S\)</span> of teams . For each
<span class="math inline">\(x\in S\)</span> , its current score <span
class="math inline">\(w_x\)</span> .</li>
<li>For <span class="math inline">\(x,y\in S\)</span> , they will need
to play <span class="math inline">\(g_{x,y}\)</span> more games .</li>
<li>For each game , winner gains <span class="math inline">\(1\)</span>
point , loser gains <span class="math inline">\(0\)</span> point . Not
consider draw case .</li>
</ul>
<p>Consider a special team <span class="math inline">\(z\)</span> ,
determine theoretically whether <span class="math inline">\(z\)</span>
can win the league ( have the highest score ) (not necessarily
unique)</p>
<p>WOLG , we can let <span class="math inline">\(z\)</span> win all
games it needs to play , and the current score of <span
class="math inline">\(z\)</span> becomes <span
class="math inline">\(m\)</span> .</p>
<p><span class="math inline">\(z\)</span> cannot win <span
class="math inline">\(\iff\)</span> <span class="math inline">\(\exists
T\subseteq S\)</span> , <span class="math inline">\(\sum\limits_{x\in
T}w_x+\sum\limits_{x,y\in T}g_{x,y}>m|T|\)</span></p></li>
<li><p>Flow Construction</p>
<p><span class="math inline">\(\iff\)</span> <span
class="math inline">\(\exists T\subseteq S\)</span> , <span
class="math inline">\(\sum\limits_{x\in T}(w_x-m)+\sum\limits_{x,y\in
T}g_{x,y}>0\)</span></p>
<p>Min-Cut Model</p>
<p>Construction :</p>
<ul>
<li>Vertices : <span class="math inline">\(V'=V\cup E\cup
\{s,t\}\)</span></li>
<li>Edges :
<ul>
<li><span class="math inline">\(\forall (u,v)\in E\)</span> , <span
class="math inline">\((s,(u,v),g_{u,v})\)</span></li>
<li><span class="math inline">\(\forall (u,v)\in E\)</span> , <span
class="math inline">\(((u,v),u,\infty)\)</span> , <span
class="math inline">\(((u,v),v,\infty)\)</span></li>
<li><span class="math inline">\(\forall v\in V\)</span> , <span
class="math inline">\((v,t,m-w_v)\)</span></li>
</ul></li>
</ul></li>
<li><p>Analysis</p>
<ol type="1">
<li><p><span class="math inline">\(\max\limits_{T\subseteq
S}\sum\limits_{x\in T}(w_x-m)+\sum\limits_{x,y\in
T}g_{x,y}>0\)</span> <span class="math inline">\(\iff\)</span> <span
class="math inline">\(\sum_{x,y}g_{x,y}-\mathtt{MinCut}>0\)</span></p>
<p>Min-Cut in <span class="math inline">\(G'\)</span> : <span
class="math inline">\((A,V'-A)\)</span> , let <span
class="math inline">\(T=A\cap V\)</span> . The min cut value is : <span
class="math display">\[
\begin{aligned}
&\quad \sum_{x\in T} (m-w_x)+\sum_{x\notin T\lor y\notin T}
g_{x,y}\\
&=\sum_{x,y}g_{x,y}-\left(\sum_{x,y\in T}g_{x,y}+\sum_{x\in
T}(w_x-m)\right)
\end{aligned}
\]</span></p></li>
<li><p><span
class="math inline">\(\sum_{x,y}g_{x,y}-\mathtt{MinCut}>0\)</span>
<span class="math inline">\(\iff\)</span> <span
class="math inline">\(z\)</span> cannot win</p>
<p>Consider as MaxFlow : greedily allocate all games , MaxFlow obviously
<span class="math inline">\(\le \sum_{x,y}g_{x,y}\)</span> .</p>
<p>If <span class="math inline">\(<\)</span> , we cannot allocate all
games that satisfies <span class="math inline">\(f_v+w_v\le m\)</span> ,
so <span class="math inline">\(z\)</span> must lose in all possibilities
.</p>
<p>If <span class="math inline">\(=\)</span> , we can find a valid
game-result allocation , so <span class="math inline">\(z\)</span> still
have possibility to win .</p></li>
</ol></li>
<li><p>Find maximum density</p>
<ul>
<li><p>Binary search : OK but slow</p></li>
<li><p>parametric diagram</p>
<p><img src="/images/posts/AD11_fig1.png" /></p></li>
<li><p>Densest Subgraph problem can be solved by parametric flow
algorithm in poly-time .</p></li>
</ul></li>
</ol>
<h3 id="maxflow-and-lp">8.4 MaxFlow and LP</h3>
<ol type="1">
<li><p>LP form of MaxFlow</p>
<p>Goal : maximize <span
class="math inline">\(\sum_{v}f_{s,v}\)</span></p>
<p>Constraints :</p>
<ul>
<li><span class="math inline">\(\forall v\in V\backslash\{s,t\}\)</span>
, <span
class="math inline">\(\sum_{u}f_{u,v}=\sum_{u}f_{v,u}\)</span></li>
<li><span class="math inline">\(\forall f_e\in E\)</span> , <span
class="math inline">\(0\le f_e\le c_e\)</span></li>
</ul></li>
<li><p>Integral Polytope</p>
<p>Claim : If <span class="math inline">\(c_e\in \mathbb Z_+\)</span> ,
<span class="math inline">\(P\)</span> is integral ( vertices of <span
class="math inline">\(P\)</span> are integral vector )</p>
<p>Proof : By FF algorithm</p>
<p>General Result :</p>
<ul>
<li>LP constraints : <span class="math inline">\(Ax\le b\)</span></li>
<li><span class="math inline">\(A\)</span> is totally unimodular matrix
<span class="math inline">\(\to\)</span> Integral Polytope</li>
</ul></li>
</ol>
<h2 id="chapter-9-randomized-algorithm">Chapter 9 Randomized
Algorithm</h2>
<ol type="1">
<li><p>Complexity</p>
<p>P , RP , ZPP , BPP</p>
<p>Conjecture : P=RP ? (current guess : YES)</p></li>
</ol>
<h3 id="maxcut-problem">9.1 MaxCut Problem</h3>
<ol type="1">
<li><p>Description</p>
<p>Given undirected <span class="math inline">\(G=(V,E)\)</span></p>
<p>Find a cut <span class="math inline">\((A,V-A)\)</span> , s.t. <span
class="math inline">\(|\{(u,v)\in E:u\in A,v\in V-A\}|\)</span> is
maximized</p></li>
<li><p>Randomized <span class="math inline">\(2\)</span>-approximation
algorithm</p>
<p><span class="math inline">\(\forall v\in V\)</span> , <span
class="math inline">\(\begin{cases}v\in A&w.p.\frac{1}{2}\\ v\notin
A&w.p.\frac{1}{2}\end{cases}\)</span></p>
<p>Analysis : <span class="math display">\[
\begin{aligned}
&\quad E[cut(A,V-A)]\\
&=\sum_{e\in E} \Pr\{e\in cut(A,V-A)\}\\
&=\sum_{(u,v)\in E} \Pr\{(u\in A,v\notin A)\lor (u\notin A,v\in
A)\}\\