It looks like Claude tokenizer handles markdown pretty efficiently, like a ## header is like 1-2 tokens. So I think the actual token savings are smaller than the byte-level numbers suggest. Where this actually matters is if maybe if you're on Haiku with big codebases where system prompt and context fight for space. On Opus? Probably not worth making your files unreadable for humans?
I've found that maintaining minimal structural cues (typically a single repeating heading) in Claude.md helps it better index information. And anecdotally a lot less GREP'ing, web searching and wasted tokens.
I also include a short instruction a the top of my file. Requiring context to be written as single line rationales with up to 1 additional line for links/refs/pointers. While explicitly forbidding "tutorial-like" explanations and blocks/pages of code.
Curious if others have experimented with this — I found the memory system files compressed more aggressively than the project CLAUDE.md because they had more prose decoration. Anyone tried a middle ground where you keep human-readable formatting in the project file but compress the memory files only?
It looks like Claude tokenizer handles markdown pretty efficiently, like a ## header is like 1-2 tokens. So I think the actual token savings are smaller than the byte-level numbers suggest. Where this actually matters is if maybe if you're on Haiku with big codebases where system prompt and context fight for space. On Opus? Probably not worth making your files unreadable for humans?
I've found that maintaining minimal structural cues (typically a single repeating heading) in Claude.md helps it better index information. And anecdotally a lot less GREP'ing, web searching and wasted tokens.
I also include a short instruction a the top of my file. Requiring context to be written as single line rationales with up to 1 additional line for links/refs/pointers. While explicitly forbidding "tutorial-like" explanations and blocks/pages of code.
Curious if others have experimented with this — I found the memory system files compressed more aggressively than the project CLAUDE.md because they had more prose decoration. Anyone tried a middle ground where you keep human-readable formatting in the project file but compress the memory files only?
[dead]