Plus LLM 0.22, the annotated release notes
Fantastic work, Simon! Apple MLX really is amazing, isn't it? Any thoughts on Llamafile and how your LLM library compares to that?
Thank you for this, Simon
I’m a Python/Django dev on Mac M1 Max (32 gb ram) and have been using Claude sonnet extensively at work via cursor
Any thoughts on how close this can get to replacing sonnet ?
I noted you wrote that llama 70b can run locally on a 64gb ram and get close to gpt 4o level
I’m thinking of perhaps some kind of divide n conquer
1. Consider upgrading to new Mac with more ram
2. Consider using your suggested setup but for easier queries
Your thoughts about these two paths
Is this an alternative to Ollama, and if so, why would one use it instead?
Great guide! I’ve personally been really enjoying running local LLMs on my Mac Studio (still M1 Max, but lots of RAM) and this just gives me more impetus to upgrade when the new ones come out with hopefully 256gb of integrated RAM…
Thanks Simon! Appreciate your work.
Fantastic work, Simon! Apple MLX really is amazing, isn't it? Any thoughts on Llamafile and how your LLM library compares to that?
Thank you for this, Simon
I’m a Python/Django dev on Mac M1 Max (32 gb ram) and have been using Claude sonnet extensively at work via cursor
Any thoughts on how close this can get to replacing sonnet ?
I noted you wrote that llama 70b can run locally on a 64gb ram and get close to gpt 4o level
I’m thinking of perhaps some kind of divide n conquer
1. Consider upgrading to new Mac with more ram
2. Consider using your suggested setup but for easier queries
Your thoughts about these two paths
Is this an alternative to Ollama, and if so, why would one use it instead?
Great guide! I’ve personally been really enjoying running local LLMs on my Mac Studio (still M1 Max, but lots of RAM) and this just gives me more impetus to upgrade when the new ones come out with hopefully 256gb of integrated RAM…
Thanks Simon! Appreciate your work.