Run Google Gemma2:2b AI on any machine

AI

A short tutorial on running Gemma2:2B on all platforms.

Desktop (Windows / macOS / Linux)

Unless you have graphics card or M series apple chip inference will be slow.

  1. Install ollama.

  2. Open a terminal or command prompt

  3. Pull & Run the Gemma 2B model:

    ollama run gemma:2b

  4. Start chatting with the model

  5. To exit, type “/bye” or press Ctrl+C

Mobile (Android / iOS / iPadOS)

It will be quite a bit slower on mobile.

Android

  1. Download the model - https://huggingface.co/bartowski/gemma-2-2b-it-GGUF/blob/main/gemma-2-2b-it-Q4_K_M.gguf
  2. Install Maid app from play store - https://play.google.com/store/apps/details?id=com.danemadsen.maid&pcampaignid=pcampaignidMKT-Other-global-all-co-prtnr-py-PartBadge-Mar2515-1
  3. Select llama.cpp if that’s not default.
  4. Click on top-right -> Model Settings -> Load Model

iOS / iPadOS

  1. Follow instructions from this site - https://llmfarm.site/
  2. After you install LLM Farm app you should download gemma2:2b Q4_K_M for fastest inference.
  3. iOS setup is more complicated because apple… I wish you luck.