How I learnt to run my own personal AI

By now most people have tried or at least heard of ChatGPT. It is now relatively easy to run your own AI instance on your own machine.

One free tool that makes this simpler is LM Studio. It is a desktop application that you can run locally on your machine. You download and then select whichever LLM (large language model) you want to use to run it. Don’t worry, LLM basically means the brains and knowledge you want to use to ask questions.

To install it, go to LM Studio and download the version for the operating system you use (MacOS, Windows or Linux) and install it.

Once it is installed, run the application.

The first thing you will need to do is download the LLM. There is a daunting array of models and it is not easy to decide which. I did a little research and I have downloaded two models to play with:

zephyr beta (q4_k_s) orca 2 (q5_k_m)

You can search for a specific model or scroll through the list. To download one, select the instance you want and click the “Download” button next to it. It might take a while to download as they tend to be between 4-8Gb.

Once loaded you will want to try it out. Click on the chat icon (3rd one down) on the left hand side. At the top there is a dropdown to select the model to use - this will load it into memory.

Once it is loaded you can start a new chat. A text box will appear where you can type your prompt and answers will appear below it.

You are now free to play.

You can also run it as a local web server so you can query it using CURL or from an API.

Note, sometimes it will occasionally stop generating responses. Usually this can be solved by reloading the model from the dropdown or worst case restarting the application. Your previously downloaded models will still be available and don’t need to be downloaded again - you just need to select the model before chatting as before.

Links

LM Studio

Random Posts

Capturing memories

A PhD student has created a way to capture conversations with hospice patients that their relatives can play back and listen to at any time later.


Read More

The accelerated pace of image processing developments

There are some amazing things happening with image processing at the moment. I remember my graphics class at university 25 years ago and the primitive, by comparison to now, image processing algorithms we had to code. Oh how things have moved on.


Read More

Fatherly wisdom

Today my daughter turns 18 years old. Wow, time flown has by.


Read More