This is exactly the type of use case TOON was designed for – thx for sharing, Jake! Interesting to hear about the drops in token usage and latency with Nova Micro, especially since the model struggled with CSV/XML for your payloads.
Yes I did. I’m using it in production w/ Amazon Nova Micro and it is performing better. We serve a ridiculous amount of ecomm traffic daily and have observed a clear drop in token usage and a decrease in ttfb from bedrock. The model did not perform well with CSVs or XML and did not even handle large compressed JSON well. For this model it is a win across the board