Skip to content

Cache exports lookups and path patterns in module specifier generation - #64724

Draft
Gavin Kline (gwkline) wants to merge 1 commit into
microsoft:mainfrom
gwkline:perf/exports-specifier-cache
Draft

Gavin Kline (gwkline) wants to merge 1 commit into
microsoft:mainfrom
gwkline:perf/exports-specifier-cache

Conversation

@gwkline

@gwkline Gavin Kline (gwkline) commented Oct 10, 2026 •

Copy link
Copy Markdown

Fixes #64721

Analysis

See #64721. When declaration emit or auto-import needs a specifier for a file:

  • tryGetModuleNameFromExports scans the package's whole exports map.
  • tryGetModuleNameFromPaths re-resolves every paths/typesVersions pattern.

Both recompute per-target work for each entry. The nodebuilder caches only the final specifier per importing file, so the cost is importing files × table size, and it sits on the emit critical path.

Fix

  • Cache. ModuleSpecifierGenerationHost gains ModuleSpecifierGenerationCache(), which returns nil to opt out. Program owns a modulespecifiers.GenerationCache that its emit host forwards to, so the cache lives exactly as long as the program. It memoizes:
    • the subpath an exports map publishes a target under, including negative results. The key is the parsed *PackageJson, the package directory and name, the target file, the resolution mode and the compiler options.
    • the resolved patterns of each paths/typesVersions table, keyed by the table and its base directory.
  • Per-lookup work. Two things that were computed once per entry are now computed once per lookup: the candidate specifiers in tryGetModuleNameFromPaths, and the target's JS, output and declaration file names in exports/imports lookups. An entry's specifier string is built only when that entry matches.
  • typesVersions table identity. GetVersionPaths returns VersionPaths by value, and GetPaths built the table lazily on each copy. So module resolution rebuilt the table on every resolution through such a package, and the table had no stable identity to cache by. It is now built once, inside the sync.Once.

Results are unchanged: the same specifiers, in the same table order. The compiler baseline suite and the fourslash import tests pass, and emitted declarations were byte-identical on the projects below. The node_modules reachability check stays outside the cached lookup. #64599 applies cleanly on top, and its new baseline passes.

New tests in modulespecifiers/cache_test.go check cached results against uncached or expected ones. Each case runs twice against one cache, so later cases are answered from entries that earlier ones stored. They cover:

  • conditions, custom conditions and resolution modes
  • a re-read package.json
  • patterns, entries with extensions, and absolute patterns resolved from different base directories
  • different package directories and names reaching the same package.json
  • two paths tables with the same base directory
  • typesVersions lookups through fresh GetVersionPaths copies, which must leave one cache entry

TestGetVersionPathsSharesTable in packagejson/packagejson_test.go checks that copies share one table. Dropping any field from either cache key, or building the typesVersions table per copy again, fails at least one of these tests. Benchmarks cover both scans.

Micro-benchmarks (M3 Pro). The "cached" row compares the new cached path with the old uncached one:

Benchmark Time per lookup Allocations per lookup
exports scan, 180 subpaths, uncached 371 µs → 298 µs 1,984 → 1,085
paths scan, 400 patterns, uncached 331 µs → 240 µs 7,201 → 6,004
paths scan, 400 patterns, cached 331 µs → 6.4 µs 7,201 → 3

Emit (M3 Pro):

Project Emit time Allocations
exports generator from the issue 0.188 s → 0.077 s 2.82M → 0.95M
paths generator from the issue 0.64 s → 0.096 s 25.2M → 1.24M
private monorepo package, declaration build (11k files) 2.48 s → 1.45 s (total 7.39 s → 6.47 s) 56.6M → 51.2M

The profiling, the patch and this description were produced with Claude Code (Claude Fable 5.1 and Claude Opus 5.5). I have read and understand the change and will handle review myself.

Copilot Checklist

I successfully ran the applicable command at the end of my session, and it completed without error:

  • npx hereby validate
  • npx hereby validate --api (for TypeScript API changes; no API changes here)

🤖 Generated with Claude Code

@typescript-automation typescript-automation Bot added the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Oct 10, 2026
Generating a specifier for a file in a package with an "exports" map
scanned every entry of the map, and lookups through a "paths" or
"typesVersions" table resolved every pattern against the table's base
directory and recomputed the candidate specifiers for each one. The
nodebuilder caches only the final specifier per importing file, so
declaration emit repeated these scans for every file that referenced
the same target, which dominated emit on builds with large exports maps
or paths tables.

- ModuleSpecifierGenerationHost gains ModuleSpecifierGenerationCache,
  which may return nil. Program owns a GenerationCache and its emit
  host forwards to it. The cache memoizes the subpath an "exports" map
  publishes a target under, keyed by the parsed package.json, package
  directory and name, target file, resolution mode and compiler
  options, and the resolved patterns of each paths table, keyed by the
  table and its base directory.
- Candidate specifiers, and the target's emitted file names for
  "exports" and "imports" lookups, are computed once per lookup instead
  of once per entry; an entry's specifier is built only when it matches.
- GetVersionPaths returns VersionPaths by value and GetPaths built the
  table lazily on each copy, so module resolution rebuilt it on every
  resolution and it had no stable identity to cache by. It is now built
  once when the version paths are selected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gwkline
Gavin Kline (gwkline) force-pushed the perf/exports-specifier-cache branch from 9b4cd93 to e66f236 Compare October 10, 2026 21:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Module specifier generation rescans package.json exports and tsconfig paths for every importing file

1 participant