Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dexsearch optimization #10536

Merged
merged 15 commits into from
Sep 17, 2024
Merged

Dexsearch optimization #10536

merged 15 commits into from
Sep 17, 2024

Conversation

larry-the-table-guy
Copy link
Contributor

@larry-the-table-guy larry-the-table-guy commented Sep 7, 2024

In short, dexsearch is was abnormally slow. A simple query like /nds fire took ~2 seconds and /nds pivot ~4 seconds on the prod server and my decent laptop.

This PR applies a few extremely simple fixes that address ~99% of the performance problems. The aforementioned queries now take ~5 milliseconds and 30 milliseconds, respectively. Far more optimizations are possible, but won't be as simple or impactful.

Results

Below are the (very scarce) performance stats for npm test, running on an i7-9750h.
These are cheaper than /nds. The first test is especially slow because of the nature of JITs.

Initial
Before
		now executing: pivot|batonpass, mod=gen8
query prep:	 54.37550900131464 ms
dex size: 804
filters total:	 129.14655699953437 ms
	stat filters:	 0.17849896475672722 ms
	egg filters:	 0.14566901698708534 ms
	type filters:	 0.1426999494433403 ms
	resist filters:	 0.13550499826669693 ms
	weak filters:	 0.14920806139707565 ms
	ability filters: 0.12753695249557495 ms
	forme filters:	 0.12921300157904625 ms
	prepare move filters: 80.43997703120112 ms
		format:   	 74.51695001125336 ms
		ruletable:	 2.823194034397602 ms
		validator:	 2.0557279251515865 ms
		pokesource:	 0.41381001472473145 ms
	move filters:	 45.508391946554184 ms
sort:	 0.6476159989833832 ms
1421 number of unique mon objects from Dex.mod(...)
804 number of above objects that pass gen filters
total run time:			 232.11053999885917 ms
․
		now executing: pivot, mod=gen8
query prep:	 0.7473939992487431 ms
dex size: 804
filters total:	 94.04799500107765 ms
	stat filters:	 0.08011595532298088 ms
	egg filters:	 0.10512598976492882 ms
	type filters:	 0.10283003747463226 ms
	resist filters:	 0.08430599421262741 ms
	weak filters:	 0.11044797673821449 ms
	ability filters: 0.08263105899095535 ms
	forme filters:	 0.0876309834420681 ms
	prepare move filters: 75.64411291107535 ms
		format:   	 71.8328619338572 ms
		ruletable:	 1.4543550126254559 ms
		validator:	 1.648287057876587 ms
		pokesource:	 0.2687230445444584 ms
	move filters:	 16.034791000187397 ms
sort:	 0.38658300042152405 ms
1421 number of unique mon objects from Dex.mod(...)
804 number of above objects that pass gen filters
total run time:			 96.38358999788761 ms
․․․․․․․․․․․
		now executing: ice, monotype
query prep:	 0.1216839998960495 ms
dex size: 873
filters total:	 494.32178499922156 ms
	stat filters:	 0.10472292080521584 ms
	egg filters:	 0.15022199600934982 ms
	type filters:	 0.8074910305440426 ms
	resist filters:	 0.12324101477861404 ms
	weak filters:	 0.11295903846621513 ms
	ability filters: 0.10627801716327667 ms
	forme filters:	 0.10835202410817146 ms
	prepare move filters: 490.29630206152797 ms
		format:   	 76.94467301294208 ms
		ruletable:	 209.55591402947903 ms
		validator:	 202.62786000221968 ms
		pokesource:	 0.504029992967844 ms
	move filters:	 0.5825789906084538 ms
sort:	 0.08564000204205513 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
total run time:			 496.44836499914527 ms
․
		now executing: ice, monotype, spe desc
query prep:	 0.1344659999012947 ms
dex size: 873
filters total:	 444.82719799876213 ms
	stat filters:	 0.09688697010278702 ms
	egg filters:	 0.1446789614856243 ms
	type filters:	 0.5860809981822968 ms
	resist filters:	 0.11443403363227844 ms
	weak filters:	 0.10183602198958397 ms
	ability filters: 0.10196587443351746 ms
	forme filters:	 0.09404192119836807 ms
	prepare move filters: 441.19286999478936 ms
		format:   	 71.15053798630834 ms
		ruletable:	 185.74913900345564 ms
		validator:	 183.20303198322654 ms
		pokesource:	 0.4670259468257427 ms
	move filters:	 0.5626690238714218 ms
sort:	 0.09177299961447716 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
total run time:			 446.4883599989116 ms

		now executing: ice, monotype, hp desc
query prep:	 0.11399000138044357 ms
dex size: 873
filters total:	 446.07669800147414 ms
	stat filters:	 0.10812501609325409 ms
	egg filters:	 0.14091401174664497 ms
	type filters:	 0.6019999720156193 ms
	resist filters:	 0.1115570142865181 ms
	weak filters:	 0.10407200083136559 ms
	ability filters: 0.11563794314861298 ms
	forme filters:	 0.10732902958989143 ms
	prepare move filters: 442.34754206985235 ms
		format:   	 70.71124800294638 ms
		ruletable:	 187.70725399255753 ms
		validator:	 182.57627097144723 ms
		pokesource:	 0.4813690446317196 ms
	move filters:	 0.5883849188685417 ms
sort:	 0.05595799908041954 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
total run time:			 447.76308299973607 ms
Latest
After
		now executing: pivot|batonpass, mod=gen8
query prep:	 51.89018200337887 ms
filters total:	 42.756921999156475 ms
sort:		 0.6007909998297691 ms
format result:	 0.09133599698543549 ms
*** entire run time:			 141.8944509997964 ms ***
*** unaccounted:			 46.55522000044584 ms ***
․
		now executing: pivot, mod=gen8
query prep:	 0.6547169983386993 ms
filters total:	 18.113839998841286 ms
sort:		 0.4578099995851517 ms
format result:	 0.06409899890422821 ms
*** entire run time:			 20.0837190002203 ms ***
*** unaccounted:			 0.7932530045509338 ms ***
․․․․․․․․․․․
		now executing: ice, monotype
query prep:	 0.13084299862384796 ms
filters total:	 0.772537000477314 ms
sort:		 0.04282499849796295 ms
format result:	 0.018077000975608826 ms
*** entire run time:			 1.8486009985208511 ms ***
*** unaccounted:			 0.8843189999461174 ms ***
․
		now executing: ice, monotype, spe desc
query prep:	 0.07613600045442581 ms
filters total:	 0.6047910004854202 ms
sort:		 0.03371300548315048 ms
format result:	 0.08656799793243408 ms
*** entire run time:			 1.2767650038003922 ms ***
*** unaccounted:			 0.47555699944496155 ms ***

		now executing: ice, monotype, hp desc
query prep:	 0.055977001786231995 ms
filters total:	 0.6282810047268867 ms
sort:		 0.02927199751138687 ms
format result:	 0.04599300026893616 ms
*** entire run time:			 1.1314250007271767 ms ***
*** unaccounted:			 0.3719019964337349 ms ***

(I go by 'ThereRNoNamesLeft' on Showdown)

No behavioral changes (other than console.log spam), this commit is just to establish a baseline and pinpoint the slow portions
Based on preliminary benchmarks, most of the time was spent getting the move validator, even for queries that don't specify a move.
More importantly, the parameters for fetching the move validator are known very early in the function and don't change during the loop.
Pulling that portion out of the loop is an easy win.
- add subcategories for filtering on move (the next optimization target)
- report unaccounted time amounts ('known unknowns')
- make grand total stand out more
Move list depends on 'alts', which does not change in the loop over mons. Minor win, but simple.
@larry-the-table-guy
Copy link
Contributor Author

Perf comparison for latest (minor) change.

Before
Before
		now executing: pivot|batonpass, mod=gen8
query prep:	 56.680244006216526 ms
filtered dex size: 804
filters total:	 52.44317799806595 ms
	stat filters:	 0.11530093848705292 ms
	egg filters:	 0.1262970045208931 ms
	type filters:	 0.11339988559484482 ms
	resist filters:	 0.1280319094657898 ms
	weak filters:	 0.12095899134874344 ms
	ability filters: 0.11294902861118317 ms
	forme filters:	 0.22467390447854996 ms
	prepare move filters: 1.363936997950077 ms
		format:   	 0.21392299979925156 ms
		ruletable:	 0.989206001162529 ms
		validator:	 0.08005800098180771 ms
		pokesource:	 0.07034800201654434 ms
		unaccounted:	 0.010401993989944458 ms
	move filters:	 46.0352219119668 ms
		get moves:	 2.3854240104556084 ms
		learn check:	 42.2000809982419 ms
		unaccounted:	 1.449716903269291 ms
sort 0.7216309979557991 ms
1421 number of unique mon objects from Dex.mod(...)
804 number of above objects that pass gen filters
*** entire run time:			 155.67729900032282 ms ***
*** unaccounted:			 45.832245998084545 ms ***
․
		now executing: pivot, mod=gen8
query prep:	 0.7357559949159622 ms
filtered dex size: 804
filters total:	 23.192718997597694 ms
	stat filters:	 0.08050889521837234 ms
	egg filters:	 0.08070703595876694 ms
	type filters:	 0.07748296111822128 ms
	resist filters:	 0.084907166659832 ms
	weak filters:	 0.08104311674833298 ms
	ability filters: 0.08629320561885834 ms
	forme filters:	 0.07892215996980667 ms
	prepare move filters: 0.16614200174808502 ms
		format:   	 0.14658299833536148 ms
		ruletable:	 0.0077700018882751465 ms
		validator:	 0.009269997477531433 ms
		pokesource:	 0.0016029998660087585 ms
		unaccounted:	 0.0009160041809082031 ms
	move filters:	 20.848288983106613 ms
		get moves:	 1.6765490919351578 ms
		learn check:	 18.15851978212595 ms
		unaccounted:	 1.0132201090455055 ms
sort 0.5495510026812553 ms
1421 number of unique mon objects from Dex.mod(...)
804 number of above objects that pass gen filters
*** entire run time:			 25.92569799721241 ms ***
*** unaccounted:			 1.447672002017498 ms ***
․․․․․․․․․․․
		now executing: ice, monotype
query prep:	 0.11871899664402008 ms
filtered dex size: 873
filters total:	 3.2516840025782585 ms
	stat filters:	 0.07280304282903671 ms
	egg filters:	 0.0771578997373581 ms
	type filters:	 0.35424505919218063 ms
	resist filters:	 0.07407596707344055 ms
	weak filters:	 0.07325991988182068 ms
	ability filters: 0.072704017162323 ms
	forme filters:	 0.07360992580652237 ms
	prepare move filters: 1.4029119983315468 ms
		format:   	 0.09722200036048889 ms
		ruletable:	 0.8441689983010292 ms
		validator:	 0.45731599628925323 ms
		pokesource:	 0.0015619993209838867 ms
		unaccounted:	 0.002643004059791565 ms
	move filters:	 0.23265302926301956 ms
		get moves:	 0.10683392733335495 ms
		learn check:	 0 ms
		unaccounted:	 0.1258191019296646 ms
sort 0.05193500220775604 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
*** entire run time:			 5.17042999714613 ms ***
*** unaccounted:			 1.748091995716095 ms ***
․
		now executing: ice, monotype, spe desc
query prep:	 0.09177500009536743 ms
filtered dex size: 873
filters total:	 2.747948996722698 ms
	stat filters:	 0.07234396040439606 ms
	egg filters:	 0.07728186994791031 ms
	type filters:	 0.2567209228873253 ms
	resist filters:	 0.07246101647615433 ms
	weak filters:	 0.07624068111181259 ms
	ability filters: 0.07200000435113907 ms
	forme filters:	 0.07326596975326538 ms
	prepare move filters: 1.0142109990119934 ms
		format:   	 0.09477400034666061 ms
		ruletable:	 0.5064290016889572 ms
		validator:	 0.4107019975781441 ms
		pokesource:	 0.0013450011610984802 ms
		unaccounted:	 0.0009609982371330261 ms
	move filters:	 0.22153691202402115 ms
		get moves:	 0.10544388741254807 ms
		learn check:	 0 ms
		unaccounted:	 0.11609302461147308 ms
sort 0.045048996806144714 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
*** entire run time:			 4.098100997507572 ms ***
*** unaccounted:			 1.2133280038833618 ms ***

		now executing: ice, monotype, hp desc
query prep:	 0.07578599452972412 ms
filtered dex size: 873
filters total:	 3.006783001124859 ms
	stat filters:	 0.07384903728961945 ms
	egg filters:	 0.09440014511346817 ms
	type filters:	 0.373492032289505 ms
	resist filters:	 0.07403897494077682 ms
	weak filters:	 0.07448684424161911 ms
	ability filters: 0.07351402193307877 ms
	forme filters:	 0.08078712970018387 ms
	prepare move filters: 1.0542949959635735 ms
		format:   	 0.09621100127696991 ms
		ruletable:	 0.516185000538826 ms
		validator:	 0.43790699541568756 ms
		pokesource:	 0.002696998417377472 ms
		unaccounted:	 0.0012950003147125244 ms
	move filters:	 0.23256384581327438 ms
		get moves:	 0.11144090443849564 ms
		learn check:	 0 ms
		unaccounted:	 0.12112294137477875 ms
sort 0.05877600610256195 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
*** entire run time:			 4.6098770052194595 ms ***
*** unaccounted:			 1.4685320034623146 ms ***
After
Before
		now executing: pivot|batonpass, mod=gen8
query prep:	 52.78931099921465 ms
filtered dex size: 804
filters total:	 48.023138001561165 ms
	stat filters:	 0.11502217501401901 ms
	egg filters:	 0.10442205518484116 ms
	type filters:	 0.113189198076725 ms
	resist filters:	 0.0974508672952652 ms
	weak filters:	 0.12177606672048569 ms
	ability filters: 0.09783709049224854 ms
	forme filters:	 0.10375013947486877 ms
	prepare move filters: 1.4203269928693771 ms
		format:   	 0.2573249936103821 ms
		ruletable:	 1.0147070065140724 ms
		validator:	 0.07130900025367737 ms
		pokesource:	 0.06653899699449539 ms
		unaccounted:	 0.010446995496749878 ms
	move filters:	 44.088605120778084 ms
		get moves:	 0.02246899902820587 ms
		learn check:	 42.81362897902727 ms
		unaccounted:	 1.2525071427226067 ms
sort 0.5674260035157204 ms
1421 number of unique mon objects from Dex.mod(...)
804 number of above objects that pass gen filters
*** entire run time:			 150.75026900321245 ms ***
*** unaccounted:			 49.37039399892092 ms ***
․
		now executing: pivot, mod=gen8
query prep:	 0.7280709967017174 ms
filtered dex size: 804
filters total:	 20.988303996622562 ms
	stat filters:	 0.07573795318603516 ms
	egg filters:	 0.08135000616312027 ms
	type filters:	 0.07815118134021759 ms
	resist filters:	 0.07936599105596542 ms
	weak filters:	 0.09821894764900208 ms
	ability filters: 0.07655186951160431 ms
	forme filters:	 0.07402002066373825 ms
	prepare move filters: 0.18907099962234497 ms
		format:   	 0.1695460006594658 ms
		ruletable:	 0.0077790021896362305 ms
		validator:	 0.009063996374607086 ms
		pokesource:	 0.001828998327255249 ms
		unaccounted:	 0.0008530020713806152 ms
	move filters:	 18.556805096566677 ms
		get moves:	 0.008464999496936798 ms
		learn check:	 17.526883997023106 ms
		unaccounted:	 1.0214561000466347 ms
sort 0.38750099390745163 ms
1421 number of unique mon objects from Dex.mod(...)
804 number of above objects that pass gen filters
*** entire run time:			 23.47478500008583 ms ***
*** unaccounted:			 1.3709090128540993 ms ***
․․․․․․․․․․․
		now executing: ice, monotype
query prep:	 0.11987200379371643 ms
filtered dex size: 873
filters total:	 3.3700819984078407 ms
	stat filters:	 0.07427005469799042 ms
	egg filters:	 0.07713989913463593 ms
	type filters:	 0.3298250213265419 ms
	resist filters:	 0.07383093982934952 ms
	weak filters:	 0.07409008592367172 ms
	ability filters: 0.07394805550575256 ms
	forme filters:	 0.07313401997089386 ms
	prepare move filters: 1.386282004415989 ms
		format:   	 0.10254500061273575 ms
		ruletable:	 0.832787998020649 ms
		validator:	 0.44645099341869354 ms
		pokesource:	 0.0018340051174163818 ms
		unaccounted:	 0.002664007246494293 ms
	move filters:	 0.10970210283994675 ms
		get moves:	 0.00045800209045410156 ms
		learn check:	 0 ms
		unaccounted:	 0.10924410074949265 ms
sort 0.03980399668216705 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
*** entire run time:			 5.092786997556686 ms ***
*** unaccounted:			 1.5630289986729622 ms ***
․
		now executing: ice, monotype, spe desc
query prep:	 0.09174399822950363 ms
filtered dex size: 873
filters total:	 2.9397659972310066 ms
	stat filters:	 0.07370498031377792 ms
	egg filters:	 0.07846103608608246 ms
	type filters:	 0.19768382608890533 ms
	resist filters:	 0.07355014979839325 ms
	weak filters:	 0.07839395850896835 ms
	ability filters: 0.07384713739156723 ms
	forme filters:	 0.1159859448671341 ms
	prepare move filters: 0.9995559975504875 ms
		format:   	 0.09128499776124954 ms
		ruletable:	 0.49932999908924103 ms
		validator:	 0.40650100260972977 ms
		pokesource:	 0.0012530013918876648 ms
		unaccounted:	 0.0011869966983795166 ms
	move filters:	 0.10992196947336197 ms
		get moves:	 0.0005569979548454285 ms
		learn check:	 0 ms
		unaccounted:	 0.10936497151851654 ms
sort 0.03881300240755081 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
*** entire run time:			 4.067071005702019 ms ***
*** unaccounted:			 0.9967480078339577 ms ***

		now executing: ice, monotype, hp desc
query prep:	 0.06570400297641754 ms
filtered dex size: 873
filters total:	 3.096519999206066 ms
	stat filters:	 0.07612402737140656 ms
	egg filters:	 0.08095891028642654 ms
	type filters:	 0.39802011102437973 ms
	resist filters:	 0.07463895529508591 ms
	weak filters:	 0.0749899223446846 ms
	ability filters: 0.07522894442081451 ms
	forme filters:	 0.07583291828632355 ms
	prepare move filters: 0.9969040006399155 ms
		format:   	 0.0922360047698021 ms
		ruletable:	 0.5020010024309158 ms
		validator:	 0.3996400013566017 ms
		pokesource:	 0.0018050000071525574 ms
		unaccounted:	 0.0012219920754432678 ms
	move filters:	 0.11649596691131592 ms
		get moves:	 0.0005770027637481689 ms
		learn check:	 0 ms
		unaccounted:	 0.11591896414756775 ms
sort 0.040344998240470886 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
*** entire run time:			 4.175630994141102 ms ***
*** unaccounted:			 0.9730619937181473 ms ***

@larry-the-table-guy larry-the-table-guy changed the title Proof-of-concept dexsearch optimization Work-in-progress dexsearch optimization Sep 8, 2024
Neither 'mod' nor 'altMoves' changes during the inner loop.
pokemonSources, which appears to act as a set, grows with each iteration, adding seemingly redundant items.
Will need to look closely at TeamValidator to identify the problem. My guess is that it's putting object references into a Set,
which, for objects other than strings, only cares about object identity.
@larry-the-table-guy
Copy link
Contributor Author

Example of issue identified in previous commit about possible leak.
Note how the same few moves keep appearing in 'restrictiveMoves'.

when the imposter is sus
		now executing: pivot, mod=gen8
query prep:	 0.7427679998800159 ms
filtered dex size: 804
altsMoves size:  7  ->  5 (after filtering for gen)
PokemonSources {
  sources: [ '8V' ],
  sourcesBefore: 0,
  sourcesAfter: 3,
  isHidden: null,
  limitedEggMoves: undefined,
  moveEvoCarryCount: 0,
  dreamWorldMoveCount: 0,
  restrictiveMoves: [
    'Flip Turn',    'Parting Shot', 'Teleport',     'U-turn',
    'Volt Switch',  'Flip Turn',    'Parting Shot', 'Flip Turn',
    'Parting Shot', 'Teleport',     'U-turn',       'Volt Switch',
    'Flip Turn',    'Parting Shot', 'Teleport',     'U-turn',
    'Volt Switch',  'Flip Turn',    'Parting Shot', 'Teleport',
    'U-turn',       'Volt Switch',  'Flip Turn',    'Parting Shot',
    'Teleport',     'U-turn',       'Volt Switch',  'Flip Turn',
    'Parting Shot', 'Teleport',     'U-turn',       'Volt Switch',
    'Flip Turn',    'Parting Shot', 'Teleport',     'U-turn',
    'Volt Switch',  'Flip Turn',    'Parting Shot', 'Teleport',
    'U-turn',       'Volt Switch',  'Flip Turn',    'Parting Shot',
    'Teleport',     'U-turn',       'Volt Switch',  'Flip Turn',
    'Parting Shot', 'Teleport',     'U-turn',       'Volt Switch',
    'Flip Turn',    'Parting Shot', 'Teleport',     'U-turn',
    'Volt Switch',  'Flip Turn',    'Parting Shot', 'Teleport',
    'U-turn',       'Volt Switch',  'Flip Turn',    'Parting Shot',
    'Teleport',     'U-turn',       'Volt Switch',  'Flip Turn',
    'Parting Shot', 'Teleport',     'U-turn',       'Volt Switch',
    'Flip Turn',    'Parting Shot', 'Teleport',     'U-turn',
    'Volt Switch',  'Flip Turn',    'Parting Shot', 'Teleport',
    'U-turn',       'Volt Switch',  'Flip Turn',    'Parting Shot',
    'Teleport',     'U-turn',       'Volt Switch',  'Flip Turn',
    'Parting Shot', 'Teleport',     'Flip Turn',    'Parting Shot',
    'Teleport',     'Flip Turn',    'Parting Shot', 'Teleport',
    'U-turn',       'Flip Turn',    'Parting Shot', 'Teleport',
    ... 3220 more items
  ],
  pomegEventEgg: null,
  babyOnly: 'porygon'
}

I'll tackle this next, could be another big, easy win.

Also, semi-fix issue identified in prior commit - the endless growth of the restrictiveMoves list.
Counting the calls to checkCanLearn helps us reason about whether the cost per call is reasonable.
use the cached ruleTable, save 100x.
@larry-the-table-guy
Copy link
Contributor Author

Before and after switching to using the cached ruletable, using '/nds pivot' as the command.
~100x from one line.

Before
		now executing: pivot, mod=base, natdex
query prep:	 0.9560710042715073 ms
filtered dex size: 1269
altsMoves size:  7  ->  7 (after filtering for gen)
PokemonSources {
  sources: [ '8V' ],
  sourcesBefore: 0,
  sourcesAfter: 3,
  isHidden: null,
  limitedEggMoves: undefined,
  moveEvoCarryCount: 0,
  dreamWorldMoveCount: 0,
  restrictiveMoves: [
    'Chilly Reception',
    'Flip Turn',
    'Parting Shot',
    'Shed Tail',
    'Teleport',
    'U-turn',
    'Volt Switch'
  ],
  pomegEventEgg: null,
  babyOnly: 'zigzagoongalar'
}
filters total:	 4210.684335000813 ms
	stat filters:	 0.1686750203371048 ms
	egg filters:	 0.21845199167728424 ms
	type filters:	 0.1814040094614029 ms
	resist filters:	 0.17194996774196625 ms
	weak filters:	 0.17302998900413513 ms
	ability filters: 0.16529518365859985 ms
	forme filters:	 0.17944318801164627 ms
	prepare move filters: 2.9677659943699837 ms
		format:   	 0.20071999728679657 ms
		ruletable:	 0.9663040041923523 ms
		validator:	 1.7963699996471405 ms
		pokesource:	 0.002299003303050995 ms
		unaccounted:	 0.0020729899406433105 ms
	move filters:	 4202.91267991066 ms
		get moves:	 0.18298999965190887 ms
		learn check:	 4198.733118206263 ms
		unaccounted:	 3.9965717047452927 ms
sort 0.783660002052784 ms
1421 number of unique mon objects from Dex.mod(...)
1269 number of above objects that pass gen filters
*** entire run time:			 4214.283986002207 ms ***
*** unaccounted:			 1.8599199950695038 ms ***
called checkCanLearn 7940 times
After
		now executing: pivot, mod=base, natdex
query prep:	 2.2799250036478043 ms
filtered dex size: 1269
altsMoves size:  7  ->  7 (after filtering for gen)
PokemonSources {
  sources: [ '8V' ],
  sourcesBefore: 0,
  sourcesAfter: 3,
  isHidden: null,
  limitedEggMoves: undefined,
  moveEvoCarryCount: 0,
  dreamWorldMoveCount: 0,
  restrictiveMoves: [
    'Chilly Reception',
    'Flip Turn',
    'Parting Shot',
    'Shed Tail',
    'Teleport',
    'U-turn',
    'Volt Switch'
  ],
  pomegEventEgg: null,
  babyOnly: 'zigzagoongalar'
}
filters total:	 29.755202002823353 ms
	stat filters:	 0.13392706215381622 ms
	egg filters:	 0.1288810446858406 ms
	type filters:	 0.12454613298177719 ms
	resist filters:	 0.11706391721963882 ms
	weak filters:	 0.11532094329595566 ms
	ability filters: 0.12929703295230865 ms
	forme filters:	 0.11631488054990768 ms
	prepare move filters: 3.304356001317501 ms
		format:   	 0.2556450068950653 ms
		ruletable:	 0.7324829995632172 ms
		validator:	 2.313129998743534 ms
		pokesource:	 0.0018050000071525574 ms
		unaccounted:	 0.001292996108531952 ms
	move filters:	 23.725674100220203 ms
		get moves:	 0.04281299561262131 ms
		learn check:	 22.219505913555622 ms
		unaccounted:	 1.46335519105196 ms
sort 0.7042810022830963 ms
1421 number of unique mon objects from Dex.mod(...)
1269 number of above objects that pass gen filters
*** entire run time:			 36.298705004155636 ms ***
*** unaccounted:			 3.5592969954013824 ms ***
called checkCanLearn 7940 times

@@ -2693,7 +2693,9 @@ export class TeamValidator {
if (!setSources.restrictiveMoves) {
setSources.restrictiveMoves = [];
}
setSources.restrictiveMoves.push(move.name);
if (!setSources.restrictiveMoves.includes(move.name)) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to change restrictiveMoves from an array to a Set. The lengths I've observed are 1 - 40, heavily weighted towards the low end. Linear scan on a small array isn't too bad, and sometimes better than a hashset. I haven't checked all the usages to see if switching to a Set is reasonable, and can't be sure that there isn't some scenario that grows restrictiveMoves to hundreds or thousands of elements.

For queries that never mention a move, a considerable chunk of time is wasted getting the objects needed for
'checkCanLearn'. I measure ~1ms savings for the relevant queries, which is often a decent percentage.
@larry-the-table-guy
Copy link
Contributor Author

stats for `npm test` after latest commit
		now executing: pivot|batonpass, mod=gen8
query prep:	 52.86700200289488 ms
filtered dex size: 804
altsMoves size:  8  ->  6 (after filtering for gen)
filters total:	 47.852364003658295 ms
	stat filters:	 0.17896609753370285 ms
	egg filters:	 0.143155999481678 ms
	type filters:	 0.14961891621351242 ms
	resist filters:	 0.1513761729001999 ms
	weak filters:	 0.1287711262702942 ms
	ability filters: 0.1273561343550682 ms
	forme filters:	 0.1352308839559555 ms
	prepare move filters: 1.4071720018982887 ms
		format:   	 0.19168899953365326 ms
		ruletable:	 1.0225059986114502 ms
		validator:	 0.11663500219583511 ms
		pokesource:	 0.06255100667476654 ms
		unaccounted:	 0.013790994882583618 ms
	move filters:	 43.45844807475805 ms
		get moves:	 0.094200000166893 ms
		learn check:	 42.00118104368448 ms
		unaccounted:	 1.3630670309066772 ms
sort 0.5518050044775009 ms
1421 number of unique mon objects from Dex.mod(...)
804 number of above objects that pass gen filters
*** entire run time:			 149.36915100365877 ms ***
*** unaccounted:			 48.0979799926281 ms ***
called checkCanLearn 4093 times
․
		now executing: pivot, mod=gen8
query prep:	 0.7021059989929199 ms
filtered dex size: 804
altsMoves size:  7  ->  5 (after filtering for gen)
filters total:	 19.674112997949123 ms
	stat filters:	 0.08176809549331665 ms
	egg filters:	 0.08343783020973206 ms
	type filters:	 0.13168099522590637 ms
	resist filters:	 0.08323197066783905 ms
	weak filters:	 0.09320704638957977 ms
	ability filters: 0.07595597207546234 ms
	forme filters:	 0.07836783677339554 ms
	prepare move filters: 0.16086000204086304 ms
		format:   	 0.13938000053167343 ms
		ruletable:	 0.009502001106739044 ms
		validator:	 0.00911799818277359 ms
		pokesource:	 0.0019310042262077332 ms
		unaccounted:	 0.0009289979934692383 ms
	move filters:	 17.435226052999496 ms
		get moves:	 0.04290100187063217 ms
		learn check:	 16.659303948283195 ms
		unaccounted:	 0.7330211028456688 ms
sort 0.3605939969420433 ms
1421 number of unique mon objects from Dex.mod(...)
804 number of above objects that pass gen filters
*** entire run time:			 22.2904329970479 ms ***
*** unaccounted:			 1.5536200031638145 ms ***
called checkCanLearn 3575 times
․․․․․․․․․․․
		now executing: ice, monotype
query prep:	 0.11851299554109573 ms
filtered dex size: 873
altsMoves size:  0  ->  0 (after filtering for gen)
filters total:	 2.0748709961771965 ms
	stat filters:	 0.07474304735660553 ms
	egg filters:	 0.08400577306747437 ms
	type filters:	 0.37189092487096786 ms
	resist filters:	 0.08028294146060944 ms
	weak filters:	 0.0742870345711708 ms
	ability filters: 0.07503896206617355 ms
	forme filters:	 0.07426407188177109 ms
	prepare move filters: 0 ms
		format:   	 0 ms
		ruletable:	 0 ms
		validator:	 0 ms
		pokesource:	 0 ms
		unaccounted:	 0 ms
	move filters:	 0.13959506154060364 ms
		get moves:	 0.026354998350143433 ms
		learn check:	 0 ms
		unaccounted:	 0.1132400631904602 ms
sort 0.07700400054454803 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
*** entire run time:			 3.9081349968910217 ms ***
*** unaccounted:			 1.6377470046281815 ms ***
called checkCanLearn 0 times
․
		now executing: ice, monotype, spe desc
query prep:	 0.07978499680757523 ms
filtered dex size: 873
altsMoves size:  0  ->  0 (after filtering for gen)
filters total:	 1.945091001689434 ms
	stat filters:	 0.0746719017624855 ms
	egg filters:	 0.0791618674993515 ms
	type filters:	 0.22400610893964767 ms
	resist filters:	 0.08054586499929428 ms
	weak filters:	 0.07442803680896759 ms
	ability filters: 0.07476413995027542 ms
	forme filters:	 0.07412984222173691 ms
	prepare move filters: 0 ms
		format:   	 0 ms
		ruletable:	 0 ms
		validator:	 0 ms
		pokesource:	 0 ms
		unaccounted:	 0 ms
	move filters:	 0.13149703294038773 ms
		get moves:	 0.01993899792432785 ms
		learn check:	 0 ms
		unaccounted:	 0.11155803501605988 ms
sort 0.03773900121450424 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
*** entire run time:			 3.037473000586033 ms ***
*** unaccounted:			 0.9748580008745193 ms ***
called checkCanLearn 0 times

		now executing: ice, monotype, hp desc
query prep:	 0.0632999986410141 ms
filtered dex size: 873
altsMoves size:  0  ->  0 (after filtering for gen)
filters total:	 2.0677170008420944 ms
	stat filters:	 0.07907605916261673 ms
	egg filters:	 0.08438802510499954 ms
	type filters:	 0.2342769354581833 ms
	resist filters:	 0.08450902998447418 ms
	weak filters:	 0.0782751813530922 ms
	ability filters: 0.07896699756383896 ms
	forme filters:	 0.07807186990976334 ms
	prepare move filters: 0 ms
		format:   	 0 ms
		ruletable:	 0 ms
		validator:	 0 ms
		pokesource:	 0 ms
		unaccounted:	 0 ms
	move filters:	 0.13780495524406433 ms
		get moves:	 0.020099997520446777 ms
		learn check:	 0 ms
		unaccounted:	 0.11770495772361755 ms
sort 0.03315500169992447 ms
2842 number of unique mon objects from Dex.mod(...)
1677 number of above objects that pass gen filters
*** entire run time:			 3.0223999992012978 ms ***
*** unaccounted:			 0.8582279980182648 ms ***
called checkCanLearn 0 times

@singiamtel
Copy link
Contributor

Wow, I never realized how expensive .getRuleTable() is. This can probably be optimized in several other places too

This is to document that they are not the bottlenecks.
We've gotten to the point where we actually *are* measuring microseconds, so the frequent calls
are too expensive. This is a good problem to have! What we can see from running npm test is
that 'filters' and 'unaccounted' still account for a majority of the time. So, there's still
room for improvement, if that's ever a serious concern.
Personally, I think working on the moves API would be more fruitful.
No more console.log or performance.now() calls. Ready to merge.
@larry-the-table-guy larry-the-table-guy changed the title Work-in-progress dexsearch optimization Dexsearch optimization Sep 11, 2024
@larry-the-table-guy larry-the-table-guy marked this pull request as ready for review September 11, 2024 22:20
@larry-the-table-guy
Copy link
Contributor Author

I think runDexSearch is now in a pretty good spot, and that this PR can stop here, while the changes are still easy to verify.

It's perfectly fine to squash these commits, many were just intended as a trail of breadcrumbs so that one can follow the reasoning and try the experiments themselves.

Comment on lines 1287 to 1288
/** @ts-expect-error validator and pokemonSource won't be undefined if there's at least one move */
if (!validator.checkCanLearn(move, dex[mon], pokemonSource) === alts.moves[move.id]) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/** @ts-expect-error validator and pokemonSource won't be undefined if there's at least one move */
if (!validator.checkCanLearn(move, dex[mon], pokemonSource) === alts.moves[move.id]) {
if (!validator.checkCanLearn(move, dex[mon], pokemonSource) === alts.moves[move.id]) {
Suggested change
/** @ts-expect-error validator and pokemonSource won't be undefined if there's at least one move */
if (!validator.checkCanLearn(move, dex[mon], pokemonSource) === alts.moves[move.id]) {
if (validator && !validator.checkCanLearn(move, dex[mon], pokemonSource) === alts.moves[move.id]) {

We prefer not using ts-expect-error if we can avoid it. Which we can here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's understandable. Here, however, if it's not initialized and it was supposed to be, this would hide the logic error. So, my thinking was that if someone did mess up the validator initialization logic, this would fail early and loudly during tests.
But I guess that during a test, this would produce wrong values anyway, and it would eventually get caught.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case you would want to do

if (!validator!.checkCanLearn(...

which is what I would recommend

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I completely agree that it's better to crash than to produce incorrect results.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or the other approach is to lazily initialise validator here if it hasn't already been initialised, something along these lines:

let validator;
...
    if (!validator) {
        ...
        validator = ...
    }
    if (!validator.checkCanLearn ...

@larry-the-table-guy
Copy link
Contributor Author

Ah, test is failing because I was trying to apply Mia's second set of changes through the web interface and forgot to add the (validator &&) part.
Sorry, I'll fix that now.

@KrisXV KrisXV merged commit ddf5848 into smogon:master Sep 17, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants