I don’t like the idea of restricting the model’s corpus further. Rather, I think it would be good if it used a bigger corpus, but added the date of origin for each element as further context.

Separately, I think it could be good to train another LLM to recognize biases in various content, and then use that to add further context for the main LLM when it ingests that content. I’m not sure how to avoid bias in that second LLM, though. Maybe complete lack of bias is an unattainable ideal that you can only approach without ever reaching it.

https://youtu.be/LRq_SAuQDec?si=0bTNdugjosLnu9-r

People complain about the total cost too get everything in The Sims, but I don’t remember anyone complaining about the total cost of every song in Rock Band.

Someone would just release a mod that replaces the ads with Monty Python sketches.

Doing more with less until we can do everything with nothing!