{"id":4173,"date":"2025-02-24T00:00:00","date_gmt":"2025-02-24T00:00:00","guid":{"rendered":"https:\/\/godofprompt.io\/blog\/2025\/02\/24\/how-to-run-ai-models-locally\/"},"modified":"2025-02-24T00:00:00","modified_gmt":"2025-02-24T00:00:00","slug":"how-to-run-ai-models-locally","status":"publish","type":"post","link":"https:\/\/godofprompt.ai\/blog\/how-to-run-ai-models-locally\/","title":{"rendered":"How to Run AI Models Locally Using Mac Terminal: A Step-by-Step Guide"},"content":{"rendered":"
Key takeaway:<\/strong> <\/p>\n

1\ufe0f\u20e3 Full Control & Privacy<\/strong> \u2013 Running AI models locally on a Mac ensures data security and independence from cloud services.
2\ufe0f\u20e3 Efficient AI Processing<\/strong> \u2013 Utilize Apple Silicon\u2019s Metal API and optimize performance with PyTorch for faster execution.
3\ufe0f\u20e3 Step-by-Step Setup<\/strong> \u2013 Install Python, create a virtual environment, and load AI models efficiently for smooth local execution.
4\ufe0f\u20e3 Troubleshooting Tips<\/strong> \u2013 Resolve common issues like DNS failures, optimize system resources, and verify Metal API for enhanced performanc.<\/p>\n<\/div>\n

Artificial intelligence has become an essential component of numerous sectors. From natural language processing to computer vision, and more. <\/p>\n

Many developers, even enthusiasts, desire to experiment with huge language models on their own devices.  <\/p>\n

When you run LLM locally on Mac, it provides you a complete control over the environment while simultaneously protecting data privacy. <\/p>\n

It also decreases reliance on external services. So, how do you configure and execute AI models locally on a Mac terminal?<\/p>\n

<\/p>\n
\"__wf_reserved_inherit\"<\/div>\n

<\/a>

Discover The Biggest AI Prompt Library<\/a> by God Of Prompt<\/figcaption><\/figure>\n

Preparation<\/strong><\/h2>\n

Make sure your Mac is updated to the latest macOS version. This will ensure compatibility with the latest tools.<\/p>\n

Update your system<\/strong><\/p>\n

– Open the Apple menu in the upper left corner.<\/p>\n

– Select System Preferences<\/em><\/p>\n

– then Software Update<\/em><\/p>\n

– Install all available updates<\/p>\n

Install Homebrew. This is a popular package manager for macOS. It makes it easy to install the tools you need.<\/p>\n

Troubleshooting possible problems<\/h3>\n

When working with locally hosted LLMs, various problems can arise. <\/p>\n

One of the most common is the DNS resolution failure<\/a> error. <\/p>\n

It can prevent you from loading models or packages. <\/p>\n

This error is often related to network issues or DNS settings on your Mac. <\/p>\n

If you’re also noticing lag or freezing, especially during high resource usage, it may be linked to broader performance issues.<\/p>\n

Heres a helpful guide from the Mac spinning wheel explained<\/a> – a common symptom when your system struggles to keep up.<\/p>\n

Installing Python and the Required Libraries<\/strong><\/h3>\n

Most AI models are developed in Python. Therefore, it is important to have an up-to-date version of this language.<\/p>\n

To install Python<\/em><\/strong><\/p>\n

– In the Terminal, type brew install python<\/p>\n

– Check the version of Python. python3 –version<\/p>\n

Create a virtual environment<\/em><\/strong><\/p>\n

It is recommended to create virtual environments to isolate projects.<\/p>\n

– Install venv<\/p>\n

python3 -m pip install –user virtualenv<\/p>\n

– Create a virtual environment<\/p>\n

 python3 -m venv myenv<\/p>\n

– Activate it<\/p>\n

source myenv\/bin\/activate<\/p>\n

Install the necessary packages<\/em><\/strong><\/p>\n

– Upgrade pip<\/p>\n

pip install –upgrade pip<\/p>\n

– Install the main libraries<\/p>\n

pip install numpy pandas torch<\/p>\n

Loading, Preparing a Model<\/strong><\/h3>\n

Find out how to run Apple large language model iOS code. <\/p>\n

To run large language models locally on your Mac, you need to download the appropriate models. <\/p>\n

Then you need to customize them.<\/p>\n

Determine which model meets your needs.<\/strong><\/p>\n

Download the model.<\/p>\n

Optimize for macOS:<\/p>\n

  – Use `torch` with Metal Performance Shaders to improve performance on Apple<\/p>\n

import torch<\/p>\n

device = torch.device(\u201cmps\u201d if torch.backends.mps.is_available() else \u201ccpu\u201d)<\/p>\n

model.to(device)<\/p>\n

\u200dRunning the model<\/strong><\/h3>\n

After you set up the environment and load the model, you can start running LLMs locally.<\/p>\n

– Create a text query<\/strong><\/p>\n

prompt = \u201cArtificial intelligence is changing the world because\u201d<\/p>\n

inputs = tokenizer(prompt, return_tensors=\u201cpt\u201d).to(device)<\/p>\n

– Run the text generation:<\/strong><\/p>\n

with torch.no_grad():<\/p>\n

    outputs = model.generate(inputs[\u201cinput_ids\u201d], max_length=50)<\/p>\n

response = tokenizer.decode(outputs[0], skip_special_tokens=True)<\/p>\n

print(response)<\/p>\n

The Apple Metal Experience. How to Speed Up Your Work<\/strong><\/h3>\n

Macs are equipped with Apple Silicon processors (M1, M2). <\/p>\n

Therefore, it is worth taking advantage of the Metal API to process AI calculations. In such a way you can improve performance<\/a> when running LLM locally on a Mac.<\/p>\n

Install PyTorch with Metal support<\/p>\n

  – Run the command<\/p>\n

pip install torch torchvision torchaudio<\/p>\n

Check the availability of Metal<\/p>\n

   – Type in Python<\/p>\n

import torch<\/p>\n

print(torch.backends.mps.is_available())<\/p>\n

If the result is True, then Metal is working successfully.<\/p>\n

Running the model with Metal<\/p>\n

device = torch.device(\u201cmps\u201d)<\/p>\n

model.to(device)<\/p>\n

Optimization for MacOS<\/strong><\/h3>\n

When running LLM locally on a Mac, it is essential to optimize resource usage. Use the following tips to do so.<\/p>\n