25:
1752:
1742:
386:
by feeding the uncompressed data to each compressor and seeing which compressor yields the highest compression. The coder with the best compression is probably the coder trained on the data that was most similar to the unknown data.
230:
250:
310:
290:
270:
439:
110:
104:, which states that any lossless data compression method must have an expected code length greater than or equal to the entropy of the source.
576:
382:
and already existing classes of data. This is done by generating an entropy coder/compressor for each class of data; unknown data is then
1244:
1055:
944:
407:
1450:
1273:
1067:
758:
1455:
1032:
459:
374:
Besides using entropy coding as a way to compress digital data, an entropy encoder can also be used to measure the amount of
1185:
1562:
1300:
1239:
1050:
1000:
823:
107:
More precisely, the source coding theorem states that for any source distribution, the expected code length satisfies
683:
668:
569:
328:
68:
46:
39:
1791:
1675:
1685:
1523:
1374:
1293:
1087:
1658:
1278:
1072:
860:
1755:
791:
1420:
1745:
1648:
1190:
748:
562:
738:
733:
1680:
1607:
1445:
1425:
1369:
1027:
818:
621:
401:
383:
355:
1781:
1690:
1631:
1557:
1405:
995:
990:
845:
688:
1695:
1268:
1062:
763:
527:
323:. If the approximate entropy characteristics of a data stream are known in advance (especially for
33:
312:
is the probability of the source symbol. An entropy coding attempts to approach this lower bound.
1786:
1636:
1007:
894:
850:
663:
646:
636:
1261:
1012:
796:
641:
50:
1533:
1665:
101:
522:
1349:
811:
773:
594:
235:
94:
358:
family of entropy coding techniques, which allows combination of the compression ratio of
8:
1580:
1471:
1430:
1415:
1384:
1379:
1288:
1195:
1128:
1097:
1082:
865:
1653:
1623:
1602:
1508:
1440:
1334:
1022:
838:
828:
723:
703:
698:
542:
465:
375:
332:
324:
295:
275:
255:
82:
1234:
1597:
1585:
1567:
1435:
1319:
1256:
1102:
1017:
973:
934:
616:
535:
500:
455:
440:"The use of asymmetric numeral systems as an accurate replacement for Huffman coding"
396:
359:
320:
530:(2003), gives an introduction to Shannon theory and data compression, including the
483:
Huffman, David (1952). "A Method for the
Construction of Minimum-Redundancy Codes".
469:
1572:
1528:
1501:
1496:
1354:
1339:
1249:
1158:
1153:
982:
715:
693:
585:
492:
447:
336:
1491:
1305:
1229:
1210:
1180:
1148:
1114:
673:
611:
1283:
1077:
806:
801:
658:
631:
548:
531:
496:
413:
363:
316:
98:
1775:
1590:
1538:
1205:
1200:
1175:
1107:
728:
626:
504:
451:
438:
Duda, Jarek; Tahboub, Khalid; Gadgil, Neeraj J.; Delp, Edward J. (May 2015).
340:
225:{\displaystyle \operatorname {E} _{x\sim P}\geq \operatorname {E} _{x\sim P}}
1711:
678:
653:
554:
418:
344:
491:(9). Institute of Electrical and Electronics Engineers (IEEE): 1098–1101.
1670:
1548:
1344:
1220:
1170:
379:
348:
1727:
1518:
1513:
1400:
1359:
1165:
1641:
1486:
1143:
327:), a simpler static code may be useful. These static codes include
1410:
884:
833:
924:
1759:
1364:
957:
904:
97:
method that attempts to approach the lower bound declared by
914:
768:
753:
743:
889:
855:
476:
292:
is the number of symbols used to make output codes and
523:
Information Theory, Inference, and
Learning Algorithms
437:
369:
315:
Two of the most common entropy coding techniques are
298:
278:
258:
238:
113:
354:
Since 2014, data compressors have started using the
304:
284:
264:
244:
224:
1773:
570:
584:
577:
563:
408:Context-adaptive binary arithmetic coding
252:is the number of symbols in a code word,
69:Learn how and when to remove this message
32:This article includes a list of general
482:
1774:
558:
18:
444:2015 Picture Coding Symposium (PCS)
13:
370:Entropy as a measure of similarity
362:with a processing cost similar to
164:
115:
38:it lacks sufficient corresponding
14:
1803:
515:
1751:
1750:
1741:
1740:
23:
16:Lossless data compression scheme
431:
219:
216:
213:
207:
201:
182:
157:
154:
151:
145:
139:
133:
1:
424:
7:
390:
10:
1808:
1632:Compressed data structures
954:RLE + BWT + MTF + Huffman
622:Asymmetric numeral systems
497:10.1109/jrproc.1952.273898
402:Asymmetric numeral systems
356:asymmetric numeral systems
1736:
1720:
1704:
1622:
1547:
1479:
1470:
1393:
1327:
1318:
1219:
1136:
1127:
1043:
991:Discrete cosine transform
981:
972:
921:LZ77 + Huffman + context
874:
784:
714:
602:
593:
95:lossless data compression
1696:Smallest grammar problem
452:10.1109/PCS.2015.7170048
272:is the coding function,
1792:Entropy and information
1637:Compressed suffix array
1186:Nyquist–Shannon theorem
53:more precise citations.
551:and H. Schwarz (2011).
485:Proceedings of the IRE
306:
286:
266:
246:
226:
1666:Kolmogorov complexity
1534:Video characteristics
911:LZ77 + Huffman + ANS
307:
287:
267:
247:
245:{\displaystyle \ell }
227:
102:source coding theorem
1756:Compression software
1350:Compression artifact
1306:Psychoacoustic model
296:
276:
256:
236:
111:
1746:Compression formats
1385:Texture compression
1380:Standard test image
1196:Silence compression
1654:Information theory
1509:Display resolution
1335:Chroma subsampling
724:Byte pair encoding
669:Shannon–Fano–Elias
446:. pp. 65–69.
333:Elias gamma coding
325:signal compression
302:
282:
262:
242:
222:
83:information theory
1769:
1768:
1618:
1617:
1568:Deblocking filter
1466:
1465:
1314:
1313:
1123:
1122:
968:
967:
536:arithmetic coding
461:978-1-4799-7783-3
397:Arithmetic coding
360:arithmetic coding
321:arithmetic coding
305:{\displaystyle P}
285:{\displaystyle b}
265:{\displaystyle d}
79:
78:
71:
1799:
1782:Data compression
1754:
1753:
1744:
1743:
1573:Lapped transform
1477:
1476:
1355:Image resolution
1340:Coding tree unit
1325:
1324:
1134:
1133:
979:
978:
600:
599:
586:Data compression
579:
572:
565:
556:
555:
509:
508:
480:
474:
473:
435:
337:Fibonacci coding
311:
309:
308:
303:
291:
289:
288:
283:
271:
269:
268:
263:
251:
249:
248:
243:
231:
229:
228:
223:
197:
196:
178:
177:
129:
128:
91:entropy encoding
74:
67:
63:
60:
54:
49:this article by
40:inline citations
27:
26:
19:
1807:
1806:
1802:
1801:
1800:
1798:
1797:
1796:
1772:
1771:
1770:
1765:
1732:
1716:
1700:
1681:Rate–distortion
1614:
1543:
1462:
1389:
1310:
1215:
1211:Sub-band coding
1119:
1044:Predictive type
1039:
964:
931:LZSS + Huffman
881:LZ77 + Huffman
870:
780:
716:Dictionary type
710:
612:Adaptive coding
589:
583:
518:
513:
512:
481:
477:
462:
436:
432:
427:
393:
380:streams of data
372:
329:universal codes
297:
294:
293:
277:
274:
273:
257:
254:
253:
237:
234:
233:
192:
188:
167:
163:
118:
114:
112:
109:
108:
75:
64:
58:
55:
45:Please help to
44:
28:
24:
17:
12:
11:
5:
1805:
1795:
1794:
1789:
1787:Entropy coding
1784:
1767:
1766:
1764:
1763:
1748:
1737:
1734:
1733:
1731:
1730:
1724:
1722:
1718:
1717:
1715:
1714:
1708:
1706:
1702:
1701:
1699:
1698:
1693:
1688:
1683:
1678:
1673:
1668:
1663:
1662:
1661:
1651:
1646:
1645:
1644:
1639:
1628:
1626:
1620:
1619:
1616:
1615:
1613:
1612:
1611:
1610:
1605:
1595:
1594:
1593:
1588:
1583:
1575:
1570:
1565:
1560:
1554:
1552:
1545:
1544:
1542:
1541:
1536:
1531:
1526:
1521:
1516:
1511:
1506:
1505:
1504:
1499:
1494:
1483:
1481:
1474:
1468:
1467:
1464:
1463:
1461:
1460:
1459:
1458:
1453:
1448:
1443:
1433:
1428:
1423:
1418:
1413:
1408:
1403:
1397:
1395:
1391:
1390:
1388:
1387:
1382:
1377:
1372:
1367:
1362:
1357:
1352:
1347:
1342:
1337:
1331:
1329:
1322:
1316:
1315:
1312:
1311:
1309:
1308:
1303:
1298:
1297:
1296:
1291:
1286:
1281:
1276:
1266:
1265:
1264:
1254:
1253:
1252:
1247:
1237:
1232:
1226:
1224:
1217:
1216:
1214:
1213:
1208:
1203:
1198:
1193:
1188:
1183:
1178:
1173:
1168:
1163:
1162:
1161:
1156:
1151:
1140:
1138:
1131:
1125:
1124:
1121:
1120:
1118:
1117:
1115:Psychoacoustic
1112:
1111:
1110:
1105:
1100:
1092:
1091:
1090:
1085:
1080:
1075:
1070:
1060:
1059:
1058:
1047:
1045:
1041:
1040:
1038:
1037:
1036:
1035:
1030:
1025:
1015:
1010:
1005:
1004:
1003:
998:
987:
985:
983:Transform type
976:
970:
969:
966:
965:
963:
962:
961:
960:
952:
951:
950:
947:
939:
938:
937:
929:
928:
927:
919:
918:
917:
909:
908:
907:
899:
898:
897:
892:
887:
878:
876:
872:
871:
869:
868:
863:
858:
853:
848:
843:
842:
841:
836:
826:
821:
816:
815:
814:
804:
799:
794:
788:
786:
782:
781:
779:
778:
777:
776:
771:
766:
761:
756:
751:
746:
741:
736:
726:
720:
718:
712:
711:
709:
708:
707:
706:
701:
696:
691:
681:
676:
671:
666:
661:
656:
651:
650:
649:
644:
639:
629:
624:
619:
614:
608:
606:
597:
591:
590:
582:
581:
574:
567:
559:
553:
552:
539:
532:Huffman coding
517:
516:External links
514:
511:
510:
475:
460:
429:
428:
426:
423:
422:
421:
416:
414:Huffman coding
411:
405:
399:
392:
389:
371:
368:
364:Huffman coding
317:Huffman coding
301:
281:
261:
241:
221:
218:
215:
212:
209:
206:
203:
200:
195:
191:
187:
184:
181:
176:
173:
170:
166:
162:
159:
156:
153:
150:
147:
144:
141:
138:
135:
132:
127:
124:
121:
117:
87:entropy coding
77:
76:
31:
29:
22:
15:
9:
6:
4:
3:
2:
1804:
1793:
1790:
1788:
1785:
1783:
1780:
1779:
1777:
1761:
1757:
1749:
1747:
1739:
1738:
1735:
1729:
1726:
1725:
1723:
1719:
1713:
1710:
1709:
1707:
1703:
1697:
1694:
1692:
1689:
1687:
1684:
1682:
1679:
1677:
1674:
1672:
1669:
1667:
1664:
1660:
1657:
1656:
1655:
1652:
1650:
1647:
1643:
1640:
1638:
1635:
1634:
1633:
1630:
1629:
1627:
1625:
1621:
1609:
1606:
1604:
1601:
1600:
1599:
1596:
1592:
1589:
1587:
1584:
1582:
1579:
1578:
1576:
1574:
1571:
1569:
1566:
1564:
1561:
1559:
1556:
1555:
1553:
1550:
1546:
1540:
1539:Video quality
1537:
1535:
1532:
1530:
1527:
1525:
1522:
1520:
1517:
1515:
1512:
1510:
1507:
1503:
1500:
1498:
1495:
1493:
1490:
1489:
1488:
1485:
1484:
1482:
1478:
1475:
1473:
1469:
1457:
1454:
1452:
1449:
1447:
1444:
1442:
1439:
1438:
1437:
1434:
1432:
1429:
1427:
1424:
1422:
1419:
1417:
1414:
1412:
1409:
1407:
1404:
1402:
1399:
1398:
1396:
1392:
1386:
1383:
1381:
1378:
1376:
1373:
1371:
1368:
1366:
1363:
1361:
1358:
1356:
1353:
1351:
1348:
1346:
1343:
1341:
1338:
1336:
1333:
1332:
1330:
1326:
1323:
1321:
1317:
1307:
1304:
1302:
1299:
1295:
1292:
1290:
1287:
1285:
1282:
1280:
1277:
1275:
1272:
1271:
1270:
1267:
1263:
1260:
1259:
1258:
1255:
1251:
1248:
1246:
1243:
1242:
1241:
1238:
1236:
1233:
1231:
1228:
1227:
1225:
1222:
1218:
1212:
1209:
1207:
1206:Speech coding
1204:
1202:
1201:Sound quality
1199:
1197:
1194:
1192:
1189:
1187:
1184:
1182:
1179:
1177:
1176:Dynamic range
1174:
1172:
1169:
1167:
1164:
1160:
1157:
1155:
1152:
1150:
1147:
1146:
1145:
1142:
1141:
1139:
1135:
1132:
1130:
1126:
1116:
1113:
1109:
1106:
1104:
1101:
1099:
1096:
1095:
1093:
1089:
1086:
1084:
1081:
1079:
1076:
1074:
1071:
1069:
1066:
1065:
1064:
1061:
1057:
1054:
1053:
1052:
1049:
1048:
1046:
1042:
1034:
1031:
1029:
1026:
1024:
1021:
1020:
1019:
1016:
1014:
1011:
1009:
1006:
1002:
999:
997:
994:
993:
992:
989:
988:
986:
984:
980:
977:
975:
971:
959:
956:
955:
953:
948:
946:
943:
942:
941:LZ77 + Range
940:
936:
933:
932:
930:
926:
923:
922:
920:
916:
913:
912:
910:
906:
903:
902:
900:
896:
893:
891:
888:
886:
883:
882:
880:
879:
877:
873:
867:
864:
862:
859:
857:
854:
852:
849:
847:
844:
840:
837:
835:
832:
831:
830:
827:
825:
822:
820:
817:
813:
810:
809:
808:
805:
803:
800:
798:
795:
793:
790:
789:
787:
783:
775:
772:
770:
767:
765:
762:
760:
757:
755:
752:
750:
747:
745:
742:
740:
737:
735:
732:
731:
730:
727:
725:
722:
721:
719:
717:
713:
705:
702:
700:
697:
695:
692:
690:
687:
686:
685:
682:
680:
677:
675:
672:
670:
667:
665:
662:
660:
657:
655:
652:
648:
645:
643:
640:
638:
635:
634:
633:
630:
628:
625:
623:
620:
618:
615:
613:
610:
609:
607:
605:
601:
598:
596:
592:
587:
580:
575:
573:
568:
566:
561:
560:
557:
550:
546:
544:
543:Source Coding
540:
537:
533:
529:
525:
524:
520:
519:
506:
502:
498:
494:
490:
486:
479:
471:
467:
463:
457:
453:
449:
445:
441:
434:
430:
420:
417:
415:
412:
409:
406:
403:
400:
398:
395:
394:
388:
385:
381:
377:
367:
365:
361:
357:
352:
350:
346:
342:
338:
334:
330:
326:
322:
318:
313:
299:
279:
259:
239:
210:
204:
198:
193:
189:
185:
179:
174:
171:
168:
160:
148:
142:
136:
130:
125:
122:
119:
105:
103:
100:
96:
92:
88:
84:
73:
70:
62:
59:December 2013
52:
48:
42:
41:
35:
30:
21:
20:
1712:Hutter Prize
1676:Quantization
1581:Compensation
1375:Quantization
1098:Compensation
664:Shannon–Fano
604:Entropy type
603:
541:
528:David MacKay
521:
488:
484:
478:
443:
433:
419:Range coding
373:
353:
345:unary coding
341:Golomb codes
314:
106:
90:
86:
80:
65:
56:
37:
1671:Prefix code
1524:Frame types
1345:Color space
1171:Convolution
901:LZ77 + ANS
812:Incremental
785:Other types
704:Levenshtein
349:Rice coding
51:introducing
1776:Categories
1728:Mark Adler
1686:Redundancy
1603:Daubechies
1586:Estimation
1519:Frame rate
1441:Daubechies
1401:Chain code
1360:Macroblock
1166:Companding
1103:Estimation
1023:Daubechies
729:Lempel–Ziv
689:Exp-Golomb
617:Arithmetic
549:T. Wiegand
425:References
384:classified
376:similarity
34:references
1705:Community
1529:Interlace
915:Zstandard
694:Fibonacci
684:Universal
642:Canonical
505:0096-8390
343:(such as
331:(such as
240:ℓ
199:
186:−
180:
172:∼
161:≥
137:ℓ
131:
123:∼
99:Shannon's
93:) is any
1691:Symmetry
1659:Timeline
1642:FM-index
1487:Bit rate
1480:Concepts
1328:Concepts
1191:Sampling
1144:Bit rate
1137:Concepts
839:Sequitur
674:Tunstall
647:Modified
637:Adaptive
595:Lossless
470:20260346
391:See also
378:between
232:, where
1649:Entropy
1598:Wavelet
1577:Motion
1436:Wavelet
1416:Fractal
1411:Deflate
1394:Methods
1181:Latency
1094:Motion
1018:Wavelet
935:LHA/LZH
885:Deflate
834:Re-Pair
829:Grammar
659:Shannon
632:Huffman
588:methods
410:(CABAC)
47:improve
1760:codecs
1721:People
1624:Theory
1591:Vector
1108:Vector
925:Brotli
875:Hybrid
774:Snappy
627:Golomb
503:
468:
458:
339:) and
36:, but
1551:parts
1549:Codec
1514:Frame
1472:Video
1456:SPIHT
1365:Pixel
1320:Image
1274:ACELP
1245:ADPCM
1235:ÎĽ-law
1230:A-law
1223:parts
1221:Codec
1129:Audio
1068:ACELP
1056:ADPCM
1033:SPIHT
974:Lossy
958:bzip2
949:LZHAM
905:LZFSE
807:Delta
699:Gamma
679:Unary
654:Range
526:, by
466:S2CID
404:(ANS)
85:, an
1563:DPCM
1370:PSNR
1301:MDCT
1294:WLPC
1279:CELP
1240:DPCM
1088:WLPC
1073:CELP
1051:DPCM
1001:MDCT
945:LZMA
846:LDCT
824:DPCM
769:LZWL
759:LZSS
754:LZRW
744:LZJB
534:and
501:ISSN
456:ISBN
319:and
89:(or
1608:DWT
1558:DCT
1502:VBR
1497:CBR
1492:ABR
1451:EZW
1446:DWT
1431:RLE
1421:KLT
1406:DCT
1289:LSP
1284:LAR
1269:LPC
1262:FFT
1159:VBR
1154:CBR
1149:ABR
1083:LSP
1078:LAR
1063:LPC
1028:DWT
1013:FFT
1008:DST
996:DCT
895:LZS
890:LZX
866:RLE
861:PPM
856:PAQ
851:MTF
819:DMC
797:CTW
792:BWT
764:LZW
749:LZO
739:LZ4
734:842
547:by
493:doi
448:doi
351:).
347:or
335:or
190:log
81:In
1778::
1426:LP
1257:FT
1250:DM
802:CM
499:.
489:40
487:.
464:.
454:.
442:.
366:.
1762:)
1758:(
578:e
571:t
564:v
545:,
538:.
507:.
495::
472:.
450::
300:P
280:b
260:d
220:]
217:)
214:)
211:x
208:(
205:P
202:(
194:b
183:[
175:P
169:x
165:E
158:]
155:)
152:)
149:x
146:(
143:d
140:(
134:[
126:P
120:x
116:E
72:)
66:(
61:)
57:(
43:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.