some perf improvements

[wilaak]

Fixes #1, #2.

Replaces the Lexer/LexerRules/Tokens object graph with a single-pass parsee of free functions in src/Parser.php. Output is byte-identical; 104 existing tests pass unchanged. Public API (Parser, Markdown) preserved.

The previous parser allocated approximately 1800 token objects per parse and cloned the Parser+Lexer tree once per block for inline re-parsing. The new implementation scans the source once with strcspn/strpos and writes HTML directly into a by-ref string buffer.

Benchmark (composer bench:tempest, opcache+JIT, same host):

input	before	after	ratio
01-small	0.338 ms	0.029 ms	11.7x
02-large	5.374 ms	0.723 ms	7.4x
01-small + hl	1.088 ms	0.890 ms	1.22x
02-large + hl	26.530 ms	18.966 ms	1.40x

[brendt]

"some" 🤣

I actually started with a variant of this approach. There are two big problems with it: maintainability and extensibility.

I wonder if extensibility is worth it though. Need to think about this.

[NickSdot]

I wonder if extensibility is worth it though.

Been working a lot with external Markdown lately. Like the ones that websites now provide for agents. Many have their own MDX components. If you want to allow MDX parsing eventually, then the extensibility is probably worth it. Not sure how others consume Markdown, but I'd assume that parsing most of the time is a (near) one-time cost, thanks to result caching.

[wilaak]

Maintainability is improved, the whole src/ could be replaced with this single file. Thousands of lines of code removed already. As for extensibility, maybe there is a middle ground here somewhere that isn't using heap allocated objects with instanceof chaning for maximum performance penalty.