How I learnt to run my own personal AI

By now most people have tried or at least heard of ChatGPT. It is now relatively easy to run your own AI instance on your own machine.

One free tool that makes this simpler is LM Studio. It is a desktop application that you can run locally on your machine. You download and then select whichever LLM (large language model) you want to use to run it. Don’t worry, LLM basically means the brains and knowledge you want to use to ask questions.

To install it, go to LM Studio and download the version for the operating system you use (MacOS, Windows or Linux) and install it.

Once it is installed, run the application.

The first thing you will need to do is download the LLM. There is a daunting array of models and it is not easy to decide which. I did a little research and I have downloaded two models to play with:

zephyr beta (q4_k_s) orca 2 (q5_k_m)

You can search for a specific model or scroll through the list. To download one, select the instance you want and click the “Download” button next to it. It might take a while to download as they tend to be between 4-8Gb.

Once loaded you will want to try it out. Click on the chat icon (3rd one down) on the left hand side. At the top there is a dropdown to select the model to use - this will load it into memory.

Once it is loaded you can start a new chat. A text box will appear where you can type your prompt and answers will appear below it.

You are now free to play.

You can also run it as a local web server so you can query it using CURL or from an API.

Note, sometimes it will occasionally stop generating responses. Usually this can be solved by reloading the model from the dropdown or worst case restarting the application. Your previously downloaded models will still be available and don’t need to be downloaded again - you just need to select the model before chatting as before.

Links

LM Studio

Related Posts

How to easily implement successful remote team retrospectives

As part of our processes we regularly hold retrospectives. We used to do this all in a room with post-it notes - a fairly traditional approach. We would set up 4 areas: Liked, Longed, Learned and Lacked and proceed to plaster post-it notes over the different sections. Once everyone had finished we would organise each section into themes and then discuss and agree actions.


Read More

What is different?

Every Monday morning as part of our stand up meeting we ask a “question of the day”. A designated victim person chooses a question that each team member gets to answer. This week the question that was asked was:


Read More

Blackout by Marc Elsberg

This story documents one scenario that could happen when the nightmare happens of a massive power cut across Europe. It starts with a number of threads and weaves them well together. It follows an Italian as he realizes it is a terrorist attack on the smart power meters. This destabilizes the power grid which takes down most of the grid across Europe. A second attack targets the power plants. This leads to a power cut across Europe that lasts days and the storyline of how they track down the cause and the perpetrators.


Read More