Tinkering with LLMs

I’ve been tinkering with LLMs.

First, I created a chatbot using all my blog posts. Then I created a model to produce blog entries based on my writing.

As I went through this process, I asked myself some questions along the way :

• Does the ML model ingest my posts & keep them? What if I’d like to remove them from the logs or training set or output for others?
• The blogbot often returned no answers for questions I thought would be straightforward. How can I ensure the response rate is 99%+ before launching?
• Which post is better : hand-written, ChatGPT, or custom-trained models? How would I judge? More views? Written in my voice with alliteration & rhetoric? Use of the & instead of “and”? Injects links to other related posts?
• When is a blogbot a better user experience than search? With search, I can know that I’ve plumbed the depths of the blog, looking at every relevant post for an answer. How do I do that with a bot?
• The latency for both is significant : 5 to 25 seconds depending on the query. Google observed 400ms increase in latency decreases traffic by 20%. Will users be more patient with bots than with search?
• What if I reimagined the home page of tomtunguz.com as a chatbot interface rather than a list of all posts? That UI would personalize the experience for each visitor & each session, but it would hamper browsing? Which is the more important use case?

I imagine many product teams are asking analogous questions about how to leverage these new models.

Within the answers to those questions lies business opportunity for startups - enabling product & engineering teams to build new product experiences with confidence.