Technologies Used

React and TypeScript are used for building the interface, and I use a NodeJS and Express backend to communicate between the frontend and backend. Ollama models are also required to commincate with the chatbot, which can be downloaded here.

Development Process & Motivation

I learned about Ollama and was amazed you could run AI models locally on your own hardware, before this I only knew of ChatGPT really. By running models locally you are able to interact with AI offline and without subscription costs. I also imagined a hypothetical scenario where you are stranded on an island, and an offline AI would be able to give you advice, and even analyze the surroundings with vision capabilities.

Challenges & Solutions

The biggest challenge for me was implementing the vision capabilities with models like LLaVa, since the image data needs to be base64-encoded.

Future Plans & Improvements

• Text-to-speech and speech-to-text are some helpful additions I could make, however the best speech-to-text by Google relies on an internet access which I want to avoid to make this truly offline.
• Multiple conversations with persistent data is another feature I'd like to add in the future.
• I plan to continue developing this with an emphasis on offline use, privacy, and affordability.