Tonight I got to jam on what’s new about building products with AI on stage at
@ctoconnection:
Learnings from building JamGPT…
1. Observability was a challenge: so we had to build internal tooling to help us observe, tweak, retry what was being sent as prompts.
2. An even bigger role for design in software development! Early on, we setup a Jupyter notebook to test system prompts which were added to the Figma file, which let the designer really craft the experience.
3. Hidden conversations: even though JamGPT looks like a chat, there is actually a totally different chat happening behind the scenes to deliver the experience we wanted. You can even farm out tasks to multiple conversations (which we considered but it added latency) - so it’s really a fun puzzle to craft a user experience.
4. Edge case testing: unlike code, you never know how it will react in all the possible cases so we had to do a lot more edge case testing in QA than we would for other features.
It was really fun to build this and can’t wait to iterate as the technology advances.