Knowledge

Efficiently updatable neural network

Source 📝

36: 512:
The neural network used for the original 2018 computer shogi implementation consists of four weight layers: W1 (16-bit integers) and W2, W3 and W4 (8-bit). It has 4 fully-connected layers,
430: 122: 652: 475:
tree. While being slower than handcrafted evaluation functions, NNUE does not suffer from the 'blindness beyond the current move' problem.
479: 81: 423: 45: 97: 761: 693: 416: 671:"Efficiently Updatable Neural-Network-based Evaluation Function for computer Shogi (Unofficial English Translation)" 519:
W1 encoded the king's position and therefore this layer needed only to be re-evaluated once the king moved. It used
708: 217: 766: 87: 756: 513: 471:, or variants thereof like the king-piece-square table. NNUE is used primarily for the leaf nodes of the 112: 599: 212: 162: 572: 232: 506: 776: 771: 520: 502: 172: 551: 545: 528: 472: 347: 322: 282: 107: 192: 127: 392: 8: 540: 464: 372: 117: 77: 70: 670: 468: 157: 102: 487: 387: 167: 624: 653:"Efficiently Updatable Neural-Network-based Evaluation Function for computer Shogi" 495: 337: 297: 516:
activation functions, and outputs a single number, being the score of the board.
486:
in 2018. On 6 August 2020, NNUE was for the first time ported to a chess engine,
312: 140: 491: 483: 460: 332: 317: 185: 144: 57: 27: 750: 402: 377: 362: 292: 227: 357: 250: 197: 272: 262: 731: 741: 342: 277: 207: 490:
12. Since 2021, many of the top rated classical chess engines such as
397: 382: 327: 302: 287: 257: 35: 202: 737: 152: 713: 678: 577: 352: 307: 267: 237: 222: 367: 524: 52: 548:- The chapter about NNUE features a visualization of NNUE. 452: 494:
have an NNUE implementation to remain competitive (with
706: 748: 570: 424: 738:NNUE evaluation functions for computer shogi 709:"official-stockfish / Stockfish, NNUE merge" 431: 417: 527:(SIMD) techniques along with appropriate 16:Neural network based evaluation function 668: 650: 749: 646: 644: 93:Efficiently updatable neural networks 22:This article is part of the series on 707:Joost VandeVondele (July 25, 2020). 662: 445:efficiently updatable neural network 641: 625:"Stockfish - Chessprogramming wiki" 13: 14: 788: 725: 571:Gary Linscott (April 30, 2021). 525:single instruction multiple data 98:Handcrafted evaluation functions 34: 734:on the Chess Programming Wiki. 700: 686: 617: 592: 564: 1: 694:"Introducing NNUE Evaluation" 557: 505:without a requirement for a 7: 534: 113:Stochastic gradient descent 10: 793: 762:Artificial neural networks 669:Yu Nasu (April 28, 2018). 651:Yu Nasu (April 28, 2018). 163:Principal variation search 501:NNUE runs efficiently on 498:as a notable exception). 451:, a Japanese wordplay on 629:www.chessprogramming.org 507:graphics processing unit 503:central processing units 455:, sometimes stylised as 521:incremental computation 173:Monte Carlo tree search 552:List of chess software 546:Stockfish chess engine 529:intrinsic instructions 283:Dragon by Komodo Chess 108:Reinforcement learning 478:NNUE was invented by 128:Unsupervised learning 46:Board representations 78:Deep neural networks 71:Evaluation functions 767:Japanese inventions 541:elmo (shogi engine) 469:piece-square tables 465:evaluation function 118:Supervised learning 103:Piece-square tables 757:Evaluation methods 482:and introduced to 158:Alpha-beta pruning 467:whose inputs are 441: 440: 168:Quiescence search 147:search algorithms 28:Chess programming 784: 719: 718: 704: 698: 697: 696:. 6 August 2020. 690: 684: 683: 675: 666: 660: 659: 657: 648: 639: 638: 636: 635: 621: 615: 614: 612: 610: 596: 590: 589: 587: 585: 568: 496:Leela Chess Zero 433: 426: 419: 338:Leela Chess Zero 38: 19: 18: 792: 791: 787: 786: 785: 783: 782: 781: 747: 746: 728: 723: 722: 705: 701: 692: 691: 687: 673: 667: 663: 655: 649: 642: 633: 631: 623: 622: 618: 608: 606: 598: 597: 593: 583: 581: 569: 565: 560: 537: 437: 408: 407: 253: 243: 242: 188: 186:Chess computers 178: 177: 148: 133: 132: 73: 63: 62: 48: 17: 12: 11: 5: 790: 780: 779: 777:Computer chess 774: 772:Computer shogi 769: 764: 759: 745: 744: 735: 727: 726:External links 724: 721: 720: 699: 685: 661: 658:(in Japanese). 640: 616: 604:Stockfish Blog 600:"Stockfish 12" 591: 562: 561: 559: 556: 555: 554: 549: 543: 536: 533: 484:computer shogi 461:neural network 439: 438: 436: 435: 428: 421: 413: 410: 409: 406: 405: 400: 395: 390: 385: 380: 375: 370: 365: 360: 355: 350: 345: 340: 335: 330: 325: 320: 315: 310: 305: 300: 295: 290: 285: 280: 275: 270: 265: 260: 254: 249: 248: 245: 244: 241: 240: 235: 230: 225: 220: 215: 210: 205: 200: 195: 189: 184: 183: 180: 179: 176: 175: 170: 165: 160: 155: 149: 139: 138: 135: 134: 131: 130: 125: 120: 115: 110: 105: 100: 95: 90: 85: 74: 69: 68: 65: 64: 61: 60: 55: 49: 44: 43: 40: 39: 31: 30: 24: 23: 15: 9: 6: 4: 3: 2: 789: 778: 775: 773: 770: 768: 765: 763: 760: 758: 755: 754: 752: 743: 739: 736: 733: 730: 729: 716: 715: 710: 703: 695: 689: 681: 680: 672: 665: 654: 647: 645: 630: 626: 620: 605: 601: 595: 580: 579: 574: 567: 563: 553: 550: 547: 544: 542: 539: 538: 532: 530: 526: 522: 517: 515: 510: 508: 504: 499: 497: 493: 492:Komodo Dragon 489: 485: 481: 476: 474: 470: 466: 462: 458: 454: 450: 446: 434: 429: 427: 422: 420: 415: 414: 412: 411: 404: 401: 399: 396: 394: 391: 389: 386: 384: 381: 379: 376: 374: 371: 369: 366: 364: 361: 359: 356: 354: 351: 349: 346: 344: 341: 339: 336: 334: 331: 329: 326: 324: 321: 319: 316: 314: 311: 309: 306: 304: 301: 299: 296: 294: 291: 289: 286: 284: 281: 279: 276: 274: 271: 269: 266: 264: 261: 259: 256: 255: 252: 251:Chess engines 247: 246: 239: 236: 234: 231: 229: 226: 224: 221: 219: 216: 214: 211: 209: 206: 204: 201: 199: 196: 194: 191: 190: 187: 182: 181: 174: 171: 169: 166: 164: 161: 159: 156: 154: 151: 150: 146: 142: 137: 136: 129: 126: 124: 121: 119: 116: 114: 111: 109: 106: 104: 101: 99: 96: 94: 91: 89: 86: 83: 79: 76: 75: 72: 67: 66: 59: 56: 54: 51: 50: 47: 42: 41: 37: 33: 32: 29: 26: 25: 21: 20: 712: 702: 688: 677: 664: 632:. Retrieved 628: 619: 607:. Retrieved 603: 594: 584:December 12, 582:. Retrieved 576: 566: 518: 511: 500: 477: 456: 448: 444: 442: 218:Deep Thought 198:ChessMachine 123:Texel tuning 92: 82:Transformers 273:CuckooChess 263:Chess Tiger 751:Categories 742:github.com 634:2020-08-18 609:19 October 558:References 473:alpha–beta 343:MChess Pro 278:Deep Fritz 208:Cray Blitz 488:Stockfish 398:Turochamp 388:Stockfish 383:SmarThink 328:KnightCap 303:GNU Chess 288:Fairy-Max 258:AlphaZero 213:Deep Blue 88:Attention 58:Bitboards 535:See also 509:(GPU). 373:Shredder 233:Mephisto 203:ChipTest 480:Yu Nasu 463:-based 459:) is a 348:Mittens 313:Houdini 153:Minimax 714:GitHub 679:GitHub 578:GitHub 573:"NNUE" 353:MuZero 333:Komodo 323:Junior 318:Ikarus 308:HIARCS 268:Crafty 238:Saitek 223:HiTech 674:(PDF) 656:(PDF) 403:Zappa 393:Torch 378:Sjeng 368:Rybka 363:REBEL 298:Fruit 293:Fritz 228:Hydra 193:Belle 141:Graph 732:NNUE 611:2020 586:2020 523:and 514:ReLU 457:ƎUИИ 449:NNUE 358:Naum 145:tree 143:and 53:0x88 740:on 453:Nue 443:An 753:: 711:. 676:. 643:^ 627:. 602:. 575:. 531:. 717:. 682:. 637:. 613:. 588:. 447:( 432:e 425:t 418:v 84:) 80:(

Index

Chess programming

Board representations
0x88
Bitboards
Evaluation functions
Deep neural networks
Transformers
Attention
Efficiently updatable neural networks
Handcrafted evaluation functions
Piece-square tables
Reinforcement learning
Stochastic gradient descent
Supervised learning
Texel tuning
Unsupervised learning
Graph
tree
Minimax
Alpha-beta pruning
Principal variation search
Quiescence search
Monte Carlo tree search
Chess computers
Belle
ChessMachine
ChipTest
Cray Blitz
Deep Blue

Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.