Sometimes it feels (only for a moment though), that there might be a bit too much work for a lone AI Director 🙂

OpenAI’s GPT Builder showed me a lot of promise when it came out and I did a lot of prototyping there. This blog’s “Rautatie Asiantuntija” and “Pohjoisen Pallonpuoliskon Puolueettominen RAU-TES tulkitsija” were made with GPT Builder. Actually it was a bit crazy, as I saw it possible to build > 1M€ value solutions with GPT Builder alone and at speed I’ve never seen in my whole career = a few hours, max week with “basic” knowledge of programming. And I mean programming in natural language / pseudocode, not some mad hacker skills.

GPT Builder’s problem was that the resulting GPT’s needed to be published in OpenAI’s GPT Store. This wouldn’t be a problem otherwise, but it meant basically no “feedback loop”, which in AI means no business. Only in typical software it would mean no business – imagine app builders building software to Apple’s app store and getting no feedback from their app users’ to improve their apps. So, in software it’s impossible to build something in long term with competitive advantage without being able to improve your solution based on customer data and feedback. In AI this effect emphasizes, as eventually you need to be able to teach your AI model the valuable data you learned from your user interactions. This improves the underlying model much more than just to be better at that single user interaction. It teaches the model patters that “align” the model as a whole for your use case. The latter means also huge resources, so the improvement is done manually at start with no possibility to teach your model – still, it’s a necessity that you get that data. And squeeze every bit of value out of it for your solution to be better.

But GPT Builder taught me some important lessons about programming AI’s in general. First of all it taught me to do proper prompt engineering. It was pretty exciting moment to find out that programming AI’s is very different compared to normal programming. I can provide the info how it’s different, but it’s so pleasant to find out yourself, that don’t wanna spoil it from you.

I’m just saying that a lot of very good “normal” programmers have hard time ever programming AI’s, as it’s completely opposite they’ve used to. Programmers usually want to have total control of the output of their creation, and the uncertainty AI brings into play is not something they very much desire. Actually it’s something they despise so much, that it might be unbearable task for some of today’s great programmers to even program AI’s due to this uncertainty, and lack of total control of their code and its outputs.

This is also one reason why everyone is building RAG’s at the moment – to minimize the hallucination of LLM’s and be as accurate as possible. So we are minimizing the very thing why modern LLM’s are so good – it’s due to their hallucination of whole Internet. Accuracy is of course a necessity for many use cases, where preciseness is appreciated. Anyway, I don’t fully see the point of focusing only on RAGs, as when preciseness is needed, then the use case is _always_ to provide links to the original docs, not to provide complete, accurate answers by the LLM. Only thing actually needed, are the correct links to original docs, which already bring the value by speeding up sometimes getting the information as much as 10 minutes in real life use cases.

Unlike many, I want to get back to prompt engineering my GPT’s. And I want to build totally new, creative use cases, and explore the blue waters out there. And I think the biggest value is in the unexplored cases.

But as time passed like 1-2 months (which is eternity in AI time), it became obvious to me, that while tools are improving for sure, you will never get the control you need of your AI app without building everything from the scratch. While tools from OpenAI etc are improving, no help is coming for anybody else than big companies. I’ve discussed with Hugging Face and OpenAI, and it’s 7-8 figures you have to throw in the table to be able to start discussion. Democracy of AI, are you kidding me.

I’m not very keen into doing stuff that’s already been solved better by other’s (like embeddings or retrieval), but as nothing’s free in this world, I’ll do it. It’s a bit funny, that GPT Builder’s natural language “code” was released yesterday 11th of May 2024 by OpenAI. Check it out. It does not solve however the retrieval etc, but it’s valuable for learning how to program your AI.

I started from quality of data, getting to know it better, categorization, hierarchization, keyword extraction, annotating, labeling, tagging etc. And most of all traditional, efficient data structures, which make it possible to do O(1) time complexity searches from the data.

And the tools definitely are improving in the meanwhile – Assistants API v2 made the Retrieval public and “proved” my case, as it has exactly same kind of traditional search alongside vector database and embeddings based search/retrieval. So their Retrieval doesn’t work optimally with only AI/LLM.

In the meanwhile it was also a bit funny watching the whole world just chunking & tokenizing their pdf’s into embeddings and dump them to LLM’s and wait them to do the rest. It should be obvious that then you hit the wall with your 60% accuracy RAG. And then start to think, wait a minute, should I do something to the data actually.

You can’t go around learning your data completely, building relationships, metadata, hierarchies, categories, tagging, annotating, labeling, finding SEO keywords, cleaning everything etc. In addition to building efficient data structures, coming up with solid search strategies and optimizing time/space complexity of your use cases (usually time complexity of search) – even without AI. This is where I’ve started practicing, and my base is now pretty solid. Of course already I’m utilizing NLP, ML and LLM in enhancing the quality of data set. But not before NOW it’s time to move to embeddings and real-time app building. Next I’m trying to break through to end2end case the easiest, but durable way – this still means in practice, now when I moved closer to front development, I had to create 5 new workspaces. There is simply too much work to do for one person, so trying to establish partnerships.

I’m building on top of Cloudflare, which should be obvious for any startup kind of business or hobby. For front I’ll be using React and Next, selection locked as of today 16.5.2024 🙂 And of course Cloudflare supports more and more Python all the time. I can bear with some TS but not wanna touch in any JS, if possible. Embeddings / LLM is obviously GPT-4 latest model due to Finnish language – no other options, really. For query analysis, and also enhancing data set (lemmatization, compound words etc Finnish morphological language at it’s finest) I’m using TurkuNLP / Hugging Face. By the way, I managed to write to my stop words 85 versions of word “see” (“nähdä” in Finnish).. is it 3 in English?

But Cloudflare ecosystem, serverless, React and Next are now sealed. And of course Python backend and OpenAI LLM – for now. BUT – of course I’m always ready to change my mind in a second, if something better comes available. Time for big clouds comes later, if it comes. But let’s see how this story continues. And be in touch, if interested in cooperation and have great passion for building AI!

GPT-4o – ei voi olla ikinä väärässä 🙂