How dMLX controls memory

Before each job, the engine calculates the memory that the job needs. The engine compares this value with the memory that it can use without a need for macOS to compress or swap other apps. The engine also keeps a reserve of memory for the system. Then it selects one of these strategies:

Strategy	When	What stays loaded
eager	a large quantity of free memory	all parts; they stay in memory for the next requests
phase-free	some free memory	image model and decoder; the engine removes the text encoder after it reads the prompt
sequential	low free memory	one large part at a time
streamed	very low free memory	one block at a time; the engine reads each block from disk immediately before it runs (~25% slower; a 1024² image in ~1.5 GB)

On Macs with less than 32 GB, FLUX.2 Klein decodes in exact bands (about 1.1 GB at 1024², not 8.6 GB). If no strategy fits, the engine refuses the job and shows a clear message. Try anyway runs the job in the mode that uses the least memory. While a job runs, the engine monitors memory pressure and swap-ins. The engine stops the job before the Mac starts to hang. The engine removes the agent from memory before each image job. Then the engine waits until that memory is free again. For more data, refer to dmlx-spec.md §8, docs/benchmarks/b1.md, b2.md and agent-16gb.md.