-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathcompression.html
539 lines (501 loc) · 31.4 KB
/
compression.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
<!DOCTYPE HTML>
<!--
Editorial by HTML5 UP
html5up.net | @ajlkn
Free for personal and commercial use under the CCA 3.0 license (html5up.net/license)
-->
<html>
<head>
<title>Neural Image & Video Compression</title>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no" />
<link rel="stylesheet" href="assets/css/main.css" />
</head>
<body class="is-preload">
<!-- Wrapper -->
<div id="wrapper">
<!-- Main -->
<div id="main">
<div id="topbar"></div>
<div class="inner">
<!-- Content -->
<section id="project">
<header class="main">
<h1>Neural Image & Video Compression
</br><span>@Disney Research</span>
</h1>
</header>
<div class="flex-box project-banner">
<div id="compression-row">
<a class="flex-image" href="https://openaccess.thecvf.com/content_CVPRW_2019/html/CLIC_2019/Campos_Content_Adaptive_Optimization_for_Neural_Image_Compression_CVPRW_2019_paper.html" target="_blank">
<img src="images/image_compression.png" alt="Image Compression">
</a>
<a class="flex-image" href="https://studios.disneyresearch.com/2019/10/27/neural-inter-frame-compression-for-video-coding/" target="_blank">
<img src="images/video_compression2.png" alt="Video Compression">
</a>
</div>
</div>
<header class="major">
<h2>Introduction</h2>
</header>
<p>
Most of us are familiar with names like MP3, JPEG, and MPEG. Although we probably know them as file extensions, they are not just that: they are, first and foremost, compression standards
<a href="#footnotes" class="tooltip bottom" data-tooltip="The internet relies on such codecs in order to deliver audio, image, and video content to our devices almost instantaneously and with very little quality loss.">1</a>.
</p>
<p>
Here, I will try to explain in simple terms the general idea behind compression algorithms,
and talk briefly about the research that I conducted with my colleagues at
<a href="https://studios.disneyresearch.com/researchlab/disney-research-zurich/" target="_blank">Disney Research.</a>
</p>
<hr class="major" />
<header class="major">
<h2>What is Compression?</h2>
</header>
<p>
Let's imagine that your friend Sarah would like to try your famous homemade vegan chocolate cake.
If you want to make sure that she tries the best possible version of the cake, it's straightforward:
you bake the cake yourself, carry the cake by hand to her house
(walking, since you don't have a car), and deliver it to her.
This works, but it is also quite inefficient: The cake is heavy and it takes time
to bring it to Sara's place, especially if you walk through busy streets.
</p>
<p>
If you are not willing to compromise on the quality of the cake, it's hard
to get something much more efficient than this.
But, even under these circumstances, there are still ways to make the cake a bit lighter
and therefore easier to carry.
One idea is to give Sarah the cake without any of the heavy
store-bought toppings like the chocolate chips and message her a list of the toppings
through <a href="https://www.signal.org/" target="_blank">Signal</a>,
including the correct brands and amounts, and with exact indications on when and where to
place them on the cake.
The catch is that this requires a bit of extra work on both sides: you need to write
all of these details, and Sarah needs to do the shopping. But with less weight
you can walk much faster and save time overall.
</p>
<p>
But if all of this still sounds too tiring, why don't you simply write down all the steps required to
bake your cake (aka, the recipe) and send it to Sarah? A recipe only occupies a few Megabytes of space, thus it is much faster to send.
But, of course, although this is a great idea, there is <a href="https://en.wikipedia.org/wiki/No_free_lunch_theorem" target="_blank">no free lunch</a> (or cake?): Now you need to accept that the end result won't be exactly like yours, especially if Sarah does not bake as well as you.
But if your goal is to save some time (actually, a lot!), it's completely worth it.
Besides, if you receive 50 other cake requests,
forwarding a message with a few Megabytes is a much better strategy than baking
50 other cakes.
</p>
<p>
Now, we already found a good solution, but let us introduce a final idea.
Suppose that Sarah and 50 other friends
all ask you for the recipe at a party (because why not?).
As you are happy to share your recipe with everyone who asks for it,
you promise them that you'll write it down in a message later on and send
it to each of them individually through Signal (you haven't discovered group chats yet).
But there is a slight problem: you realize that you only have 30 MB left on your internet budget,
so you need to save as much as space as you can!
An idea to overcome this challenge is to take advantage of the fact that, at that moment, you can
share some information with each other; for example, you can agree that “One tablespoon” will
be written as “1 Tbsp”, “half teaspoon” as “1/2 tsp”, “1 pack of oatmilk
from <a href="https://www.oatly.com/en" target="_blank">Oatly</a>” as
“1 milk” (no, I don't receive any money from them 😄), etc.
This allows you to shorten the size of your recipe and fall within your internet plan.
<br>
</p>
<h3>Back to Science</h3>
<p>
I will now frame this analogy in the language of compression.
The idea here is that you're the <i>encoder</i>—you encode the cake into a <i>compressed message</i>—, and Sarah is the <i>decoder</i>—she takes the compressed message and attempts her best <i>reconstruction</i> of the cake
from it.
</p>
<p>
In the first version, the cake was not compressed at all. Sending the complete cake
gave you guarantees of pristine quality, but also required a lot of time.
Sending uncompressed content over the internet is usually prohibitive:
If you could request an uncompressed film from Disney Plus
<a href="#footnotes" class="tooltip top" data-tooltip="I am using Disney Plus for the streaming examples
because these projects are the result of joint work with my colleagues at Disney—whom I love—and I also don't want to get into legal trouble 😅.">2</a>
at 4K resolution, they would have to send you the RGB values of each of the 8,294,400 pixels of each of the 130,000 frames of the film, which gives you a total of <b>3.43 Terabytes</b> of information to download!
Just like our car roads, internet “roads” do not have
unlimited space (bandwidth) and can get quite congested, so you'd certainly have to wait ⏱ for a good while before being able to watch the film.
</p>
<p>
In the second version, you were able to carry a lighter cake at the small
expense of a Signal message with the list of toppings. This required some additional
work on both sides, but because the time needed to carry the cake
from your place to hers was so much reduced, you saved time overall.
Here, the compressed message was the cake without toppings
plus the list; from these two components, Sarah was able to perfectly reconstruct the cake,
provided that she followed the instructions correctly.
Because you were able to save space without loosing any quality,
you performed <i>lossless compression</i>.
</p>
<p>
Lossless compression can only go up to a certain point, though.
The alternative is to allow some loss of quality
for the sake of saving a space and reducing <a href="https://en.wikipedia.org/wiki/Transmission_time" target="_blank">transmission time</a>.
This is called as (you guessed it...) <i>lossy compression</i>.
The final quality of the cake will depend on how well you can summarize the recipe and how well Sarah can reconstruct it.
One crucial point here is to strike a balance between the space occupied—which impacts the transmission time—and the reconstruction quality:
if, on the one hand, your recipe is very detailed, it will be very long and take a lot of time to send;
if, on the other hand, you summarize your recipe in very few steps, you'll be able to send it faster
but the quality of the end result will suffer.
For example, a film at 8K resolution will have almost-perfect quality
but be quite heavy, and the same film at 240p will download almost instantly,
but its quality will be very low.
This fundamental trade-off is called the
<a href="https://en.wikipedia.org/wiki/Rate%E2%80%93distortion_theory" target="_blank">Rate-Distortion curve</a> (you can see one such curve on the <a href="#compression-row">top-left image on top</a>): It tells us that if you wish to write down your chocolate cake recipe in two lines, you can,
but don't except your friend to come up with anything similar to your homemade cake.
A good way to compare compression algorithm is by comparing the reconstruction quality
for the same recipe length (as
in the <a href="#compression-row">aforementioned image</a>).
</p>
<p>
As is clear by now, a downside of compression is that it requires a bit of “local” work
from both the encoder and the decoder.
It is fair to ask whether local processing time might not become comparable to
transmission time. Since the point of compression is to save time overall, this
is intendedly not the case;
in fact, most often, encoding and decoding times
are negligible compared to the transmission speed, so the latter is the real bottleneck.
But it's important to mention that, although low processing time is a definite requirement
for the decoder, it is not so much so for the encoder,
because the latter only needs to do the job of creating each recipe once,
and it can do this before any requests are made.
Once a recipe is created, it can be stored somewhere so that, when it is requested,
it can be copied and forwarded to the receiver.
(For example, Disney Plus is not encoding films into “recipes” every time you click
on a film— they just send you the “recipe” that is already stored in their
servers.)
</p>
<p>
The last scenario above is just a way of depicting the way encoders and decoders work
in relation to each other:
if the encoder performs an operation (<i>e.g.</i>, replacing “One tablespoon” with “1 Tbsp”),
the decoder will have to do the inverse (<i>e.g.</i> replacing “1 Tbsp” with “One tablespoon”).
Roughly speaking, decoders mirror the operations of encoders.
</p>
<h3>Video Compression</h3>
<p>
Here, I will briefly explain how compression can be applied to video.
</p>
<p>
Videos are a sequence of frames (pictures). Since, there are usually 24 or 30 frames
per second (fps), the visual differences between frames are usually very small
<a href="#footnotes" class="tooltip top" data-tooltip="We say that there is a lot of redundancy in the signal.">3</a>
(an exception is when
there is scene cut).
So if we consider, for example, a 2-second static scene of a landscape,
a simple way to compress is to encode the first frame in the scene and
then send a small piece of information to the receiver that
says “The next 47 frames are the same”.
Another very simple way to perform lossy compression of a video is to just drop 1 out of 3 frames,
and send a piece of info to the receiver that says “repeat each of these frames 3 times”.
This gives you a <a href="https://www.youtube.com/watch?v=kA2XrXeHSRg" target="_blank">stop motion film</a>,
which is often used in animation for aesthetic purposes. Because close frames are so correlated,
and our brains can fill in the blanks,
doing so still gives you a fairly good—but not seamless—viewing experience.
</p>
<p>
Of course, the way actual video codecs work is much more sophisticated.
One technique that is used by all video codecs is
<a href="https://en.wikipedia.org/wiki/Motion_compensation" target="_blank">
motion compensation</a>.
The idea here is to encode key frames in a film, called intra-frames, on their own (<i>i.e.</i>,
without using the information of any other frames),
and then just encode the differences in motion of other frames relative to the intra-frames.
After reconstructing the intra-frames on their own, the decoder can then reconstruct the
remaining frames by “adding” their motion information to the intra-frames.
In this way, if an intra-frame is encoded at the beggining of an almost-static scene,
very little information is needed to reconstruct the following frames
because very little motion is present
<a href="#footnotes" class="tooltip bottom" data-tooltip="Motion information has a much lower
entropy (cf. footnote) than full frames, which means it is much easier to compress.">4</a>.
</p>
<hr class="major" />
<header class="major">
<h2>The Projects, Explained</h2>
</header>
<p>
Now, we're ready to dive into my projects on image and video compression.
For this simple explanation, I will continue using the cake analogy.
Both works are variations of neural end-to-end compression algorithms,
so we'll start with that.
</p>
<h3>Neural End-to-End Compression</h3>
<p>
Neural end-to-end compression can be likened
to you and Sarah going on a cooking retreat together; during that time,
you take a selection of cakes, write down the recipes, and
Sarah tries to bake them. As the retreat progresses, you both improve your skills
over time by <b>learning jointly</b> from your mistakes, that is, by comparing
the original cake with Sarah's version and offsetting the differences.
You gradually improve your recipe-writing skills, and Sarah her
recipe-reading-and-interpreting and baking skills.
This process is <b>neural/AI-driven</b> because the encoder and decoder learn from the
data, and it is <b>end-to-end</b> because it works solely by comparing the input (your cake)
with the output (Sarah's cake), with no other mechanism at play.
</p>
<p>
(Now, get ready for some Sci-Fi...)
</p>
<p>
After the retreat, both you and Sarah <a href="https://en.wikipedia.org/wiki/Mind_uploading" target="_blank">upload your minds</a>
into a cyborg and switch off the cyborgs' training mode:
your cyborg cannot improve its recipe-writing skills and Sarah's
cannot improve its reconstruction skills.
(Just like ChatGTP—apparently, humanity's cyborg 😅—has
its training mode switched off since its creation in 2021.)
Once you are back home and you have another cake to send (one that you haven't
necessarily experimented with during the retreat),
you ask your cyborg to write down the recipe
and send it to Sarah, and she asks her cyborg to bake the cake from it.
Having trained together for two weeks, the results will be much more satisfying
than if you had not.
</p>
<p>
Notice that now we have separated the sender (you) from the encoder (your cyborg), and
the receiver (Sarah) from the decoder (her cyborg). (In streaming terms, the
sender might be Disney Plus, the encoder a piece of software in their servers,
the decoder a piece of software in your laptop, and the receiver you—the end user.)
These distinctions are important to explain that both the sender and the receiver
can have access to the encoder and decoder. In terms of our analogy, this means that
<b>both you and Sarah can have a clone of each other's cyborg at your respective places</b>
(apart from your own cyborgs).
</p>
<h3>The First Project: Content-Adaptive Neural Image Compression</h3>
<p>
<!-- We would like an encoder to be able to take in any cake and summarize it into a recipe,
and same thing goes for the encoder in the opposite direction. -->
We're at the point where both of your cyborgs have finished their training together
and you have a new cake to send.
For this algorithm, it is necessary that apart from your own cyborg, you also have a clone
of Sarah's cyborg at your place. Assume this to be the case.
</p>
<p>
Let's consider why it is, in the case of the decoder, mandatory, and in the case of the
encoder, a good idea to switch the cyborgs' learning modes off.
</p>
<p>
First, the decoding cyborg that is at Sarah's place does not know
which cake actually originated the recipe (that's its job to guess), so it cannot use the
differences to improve itself—unlike when you were both training
together in your cooking retreat. Hence, the decoder (both hers and the
clone that you have) needs to be fixed.
</p>
<p>
What about your own cyborg?
Your cyborg does have access to both the original cake and to the end result
because you have a copy of Sarah's decoding cyborg that can bake the cake exactly
like hers.
So, in theory, you could leave your cyborg's
training mode on and run the process of recipe-writing-and-baking
a few times for each new cake so that your cyborg can adjust itself to
create a better recipe for that specific cake.
This would make the compression <b>content-adaptive</b> (or cake-adaptive).
The problem with this approach is that it does not give you reliably good results.
It might well be the case that if you start tweaking your cyborg to
improve the reconstruction quality for a specific cake, it will worsen it
for other cakes, because of over-tweaking (aka
<a href="https://en.wikipedia.org/wiki/Overfitting" target="_blank">overfitting</a>).
Moreover, if you give your still-in-training cyborg the same recipe at different times,
it will give you different results and one of them might
be much worse than if you had decided to switch off its learning mode after the cooking
retreat. (Imagine that, at some point, you download a film
on Disney Plus with good quality, and a month later the quality is much worse
with the same exact bandwidth...)
</p>
<p>
<i>
What if there was a way to magically tweak the recipe itself without
having to modify neither the encoder nor the decoder
while mantaining the length of the recipe
(so that the transmission time is not increased) and having the assurance
that the quality of the end result will be, at best, much better, and, at worst,
the same?
</i>
</p>
<p>
This is precisely what we did in this project.
The sender (you) can run the process of recipe-writing-and-baking
a few times for each new cake and simply tweak the recipe (which is now a little
learning cyborg itself), without modifying the encoder and decoding cyborgs.
Because the recipe is much less complicated than the cyborgs, it is much faster
to update, and because it occupies very little space,
you can save the original recipe
after the first encoding just in case the tweaked one
does not give you better results.
The downside is that this takes extra work and time on the sender side.
However, if what matters is the time required to send the recipe
when it is requested and not as much the time that it takes to write it down,
it's worth it! As mentioned previously, this is usually the case:
for example, when Disney Plus decides to launch a new film, it doesn't really matter
if they take an hour more to compress it than usual if the end-quality is improved,
as long as the time it takes to send it when users ask for it is not increased.
Again, the recipe only needs to be created once; after that, it can then be copied and
forwarded as much as necessary.
</p>
<p>
The <a href="#compression-row">top-left image on top</a> shows how our content-adaptive image compression method compares to modern image codecs like
<a href="https://en.wikipedia.org/wiki/Better_Portable_Graphics" target="_blank">BPG</a> and
<a href="https://en.wikipedia.org/wiki/WebP" target="_blank">WebP</a>, and to the baseline neural
compression algorithm without adaption.
</p>
<h3>The Second Project: Neural Inter-Frame Video Compression</h3>
<p>
This project incorporates many different ideas. Here, I will only give the general
picture and explain the concept of latent-space residual encoding (don't mind this).
For this algorithm, Sarah also needs a clone of your cyborg at her place, so that
both you and Sarah have access to the encoder and the decoder.
Assume this to be the case. Now, the process is a bit more convoluted,
but let's see it carefully.
</p>
<p>
This is what you, the sender, needs to do to create the compressed message for a new cake:
First, you ask your cyborg to write down the recipe for the cake.
Second, you ask your version of Sarah's cyborg to bake its flawed version of the cake
from that recipe.
Third, you make a second request to your cyborg, asking it to
write down the recipe for the cake Sarah's cyborg just baked.
(Just follow me here 😅.)
Ok, so now you have two recipes: the original one and the one for Sarah's flawed cake.
The last thing you need to do is to check the
differences between the two recipes and write down an errata (<i>i.e.</i>, a side recipe)
to correct for the mistakes.
Finally, you can send a compressed message to Sarah which consists of the original recipe
together with the errata.
</p>
<p>
Now, this is what Sarah, the receiver, needs to do to reconstruct the cake from
the original recipe and the errata:
First, she asks her cyborg to bake the cake from the recipe.
Second, she asks her version of your cyborg to write down the recipe from the flawed
cake that your cyborg just baked. (These two steps are the same as your 2nd and 3rd steps).
Third, she takes the most recent recipe, reads the errata and uses it to correct
the new recipe, thus generating a corrected version of the recipe.
Finally, she asks her cyborg to bake the final cake from the corrected recipe.
</p>
<p>
That's it! Thank you for reading this far.
</p>
<p>
The <a href="#compression-row">top-right image on top</a> shows how our video compression method compares with standard codecs such as <a href="https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding" target="_blank">HEVC/H.256</a>.
</p>
<hr class="major" />
<h2>Publications</h2>
<div id="disney-pubs" class="pubs">
</div>
<hr class="major" />
<h2>Patents</h2>
<div id="patents" class="pubs">
</div>
<hr class="major" />
<h2>Impact</h2>
<p>
We achieved state-of-the-art results at the time of publication with both works,
and together they led to the registration of three US patents
[<a href="https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/10972749" target="_blank">1</a>,
<a href="https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/11012718" target="_blank">2</a>,
<a href="https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/11057634" target="_blank">3</a>].
The work on video compression has been cited over 150 times, including in publications from
<a href="https://openaccess.thecvf.com/content/CVPR2021/papers/Wang_One-Shot_Free-View_Neural_Talking-Head_Synthesis_for_Video_Conferencing_CVPR_2021_paper.pdf" target="_blank">
Nvidia</a>,
Google [<a href="https://openaccess.thecvf.com/content_CVPR_2020/papers/Agustsson_Scale-Space_Flow_for_End-to-End_Optimized_Video_Compression_CVPR_2020_paper.pdf" target="_blank">1</a>,
<a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9242247" target="_blank">2</a>],
and
<a href="https://proceedings.neurips.cc/paper_files/paper/2021/
file/96b250a90d3cf0868c83f8c965142d2a-Paper.pdf" target="_blank">
Microsoft</a>.
The work on image compression has been cited over 40 times, including in publications from
<a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9242247" target="_blank">Google</a>,
<a href="https://arxiv.org/abs/2103.03123" target="_blank">University of Oxford</a>,
<a href="https://proceedings.neurips.cc/paper_files/paper/2020/file/
066f182b787111ed4cb65ed437f0855b-Paper.pdf" target="_blank">
UC Irvine</a>, <!-- (IMPORTANT) -->
<a href="https://openreview.net/pdf?id=IDwN6xjHnK8" target="_blank">
Qualcomm</a>,
<a href="https://arxiv.org/pdf/2008.09180.pdf" target="_blank">
Uber</a>, <!-- (CLOSE TO US) -->
<a href="https://openaccess.thecvf.com/content/CVPR2021W/CLIC/papers/Zhao_A_Universal_Encoder_Rate_Distortion_Optimization_Framework_for_Learned_Compression_CVPRW_2021_paper.pdf" target="_blank">
Microsoft</a>, <!-- (Important) -->
Nokia
[<a href="https://arxiv.org/pdf/2007.16054.pdf" target="_blank">1</a>,
<a href="https://arxiv.org/pdf/2108.09992.pdf" target="_blank">2</a>], and
<a href="https://openaccess.thecvf.com/content_CVPRW_2020/papers/w7/Guo_Variable_Rate_Image_Compression_With_Content_Adaptive_Optimization_CVPRW_2020_paper.pdf" target="_blank">
Huawei</a>.
</p>
<!-- Publications and Patents -->
<hr class="major" />
<h2>Tech Stack</h2>
<p>
Here I outline a few technologies that I used in these projects.
<ul>
<li><a href="https://www.python.org/" target="_blank">Python</a>: Programming language.</li>
<li><a href="https://pytorch.org/" target="_blank">Pytorch</a>: Deep learning framework.</li>
<li><a href="https://www.gnu.org/software/bash/" target="_blank">Bash</a>: Unix shell and command language.</li>
</ul>
</p>
<hr class="major" />
<h2>Acknowledgments</h2>
<p>
I would like to thank my Disney supervisors, Aziz Aziznejad and Christopher Schroers,
and my Disney colleagues Simon Meierhans (who gave the original idea for the first
project), Pierre, and Natalie.
</p>
<hr class="major" />
<div id="footnotes">
<h2>Footnotes</h2>
<p>
[1] More sophisticated standards have since taken the place of these older ones.
Some examples in video (which are based on MPEG) are H.265/HEVC, VP9, and AV1.
The latter is used by Netflix.
</p>
<p>
[2] I am using Disney Plus for the streaming examples
because these projects are the result of joint work with my colleagues at Disney—whom I love—and
I also don't want to get into legal trouble 😅.
</p>
<p>
[3] We say that there is a lot of redundancy in the signal.
</p>
<p>
[4] Motion information has a much lower <a href="https://en.wikipedia.org/wiki/Entropy_(information_theory)" target="_blank">entropy</a>
than full frames, which means it is much easier to compress.
</p>
</div>
</section>
</div>
<div id="footer"></div>
</div>
<!-- Sidebar -->
<div id="sidebar">
<div class="inner">
<!-- Menu -->
<nav id="menu">
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="about.html">About</a></li>
<li>
<a class="opened active" href="projects.html">Projects</a>
<ul>
<li><a href="germano.html">Healthcare IT Consultancy</a></li>
<li><a id="current" href="compression.html">Neural Image & Video Compression</a></li>
<li><a href="radiobooks.html">AI Text-to-Speech App</a></li>
<li><a href="htv.html">Hessian-Schatten Total Variation</a></li>
<li><a href="deepsplines.html">Deepsplines</a></li>
<li><a href="mahayana.html">Ethics in Mahayana Buddhism</a></li>
<li><a href="madhyamaka.html">Communicating Emptiness</a></li>
</ul>
</li>
<li><a href="pubs.html">Publications</a></li>
<li><a href="recommendations.html">Recommendations</a></li>
</ul>
</nav>
</div>
</div>
</div>
<!-- Scripts -->
<script src="assets/js/jquery.min.js"></script>
<script src="assets/js/browser.min.js"></script>
<script src="assets/js/breakpoints.min.js"></script>
<script src="assets/js/util.js"></script>
<script src="assets/js/main.js"></script>
</body>
</html>