Apple Silicon memory and performance for LLM inference

What unified memory changes

MLX uses Apple Silicon's shared memory architecture. The CPU and GPU can work with arrays in the same memory without copying them between separate CPU and GPU memory pools. This is useful for large model weights, but it does not turn installed RAM into a promise of generation speed. MLX unified memory documentation

macOS and other applications also need memory. Model execution adds temporary allocations, and a conversation's stored attention state can grow with context. Read a catalog's memory requirement as a service configuration requirement, not as the size of the model download.

A first estimate for model weights

For a dense model with 27 billion parameters, four bits per parameter gives a raw weight estimate of 13.5 billion bytes, about 12.6 GiB. Eight bits gives 27 billion bytes. Sixteen bits gives 54 billion bytes. These are arithmetic estimates before quantization metadata, non-quantized tensors, working memory and context state.

That explains why a 16 GB download and a 24 GB memory requirement can both be correct. It does not prove that every 27B model will run in 24 GB, or that two quantizations will produce equivalent answers. Use the requirements for the exact model repository and the 4-bit versus 8-bit evaluation guide to test the quality your application needs.

The reviewed PROXIES.SX requirements

The September 11 catalog lists Qwen3.8 27B in three configurations: 4-bit at 24 GB required memory and 16,384 maximum context; 8-bit at 48 GB and 32,768 context; bf16 at 96 GB and 32,768 context. These are API fields, not independent benchmark results. The model guide includes the exact IDs and repositories.

The reviewed tiers now correspond to the same three 27B configurations. Each machine still needs benchmark verification and supplier approval. Verify a catalog model and an approved listing together; installed memory alone does not determine the admitted tier.

Mac mini, Mac Studio or MacBook Pro

For an existing Mac mini or Mac Studio, check its configured memory rather than assuming that every machine in the product line has the same capacity. A permanently connected desktop avoids a laptop's lid, battery and travel interruptions. Leave room for airflow and measure power under the workload you will actually serve.

A MacBook Pro can be useful if it can stay powered and available for the rental. If it is also your daily work machine, account for competing memory and GPU use. The provider guide recommends desktop Macs and does not recommend the fanless MacBook Air for sustained serving. Hardware guidance

Read memory pressure while the workload runs

On a Mac you operate, open Activity Monitor's Memory tab while serving a representative workload. Apple describes memory pressure as a combination of free memory, swap rate, wired memory and cached files. Record pressure and swap alongside the response timings. One free-memory number before loading the model misses the conditions under load. Apple memory monitoring

Repeat the observation with a longer prompt and the simultaneous requests you intend to serve. If pressure rises while responses slow, reduce the workload and inspect the model's configuration before committing that machine to a rental. This is a diagnostic comparison, not a universal threshold for every Mac.

Context state is commonly called the KV cache, short for key-value cache. It stores attention information used during generation. Its size depends on the architecture, runtime, sequence lengths and active requests. Quantizing model weights does not automatically apply the same precision to that cache. The quantization guide separates those choices.

For renters, these host observations may be unavailable through the managed API. Mark them unknown and evaluate the endpoint's behavior directly. Providers can add host measurements to the benchmark worksheet.

Separate fitting a model from serving a customer

Loading a model proves that it fits that runtime configuration. A useful service also needs acceptable response time, answer quality and recovery from interruption. Test long prompts and several simultaneous requests instead of relying on the fastest short completion.

Record the machine's configured RAM, runtime version, model revision and power conditions with the result. The benchmark guide supplies a worksheet. If your aim is rental income, test existing hardware first and put measured operating costs into the earnings calculation.