So many news organizations are investing in collaboratively exploring generative AI and what it means for their journalism missions and businesses, in some cases with the support of major funding partners. I’m excited to see.
Our industry has an unfortunate history of becoming defensive and reluctantly forced to adapt in the face of disruptive emerging technologies. But this time, I feel like we’re making a welcome move. wonderful.
Every news organization with the means to experiment with these technologies, from the New York Times and Washington Post to nonprofit news outlets to chains like Hearst and McClatchy to independent community organizations like ours. It looks like they’re starting a small team. Good luck everyone. I can’t wait to see what happens.
My prediction for 2025 is that many of these teams will be focused on pressing real-world problems and quick wins. Between the demand for grants and the speculative nature of these investments by media companies, there will naturally be pressure to demonstrate tangible success quickly. That’s great! There are many low-hanging fruit. It’s a great place to start.
However, my hope is that it doesn’t stop there. The world of AI is vast and includes much more than the generative models that have captured the public’s imagination. We can do more than just build wrappers around tools like ChatGPT (which may be valuable). We should also work from first principles and find the time and energy to invest in breaking new ground.
In that spirit, I wanted to highlight a few areas of research and development that, to me at least, seem worthy of industry attention. Perhaps they would be good areas for collaboration among ourselves or with academic and industry partners. But at least they provide food for thought as we begin to explore this field in earnest as we head into 2025.
Benchmarking and Evaluation: Good for measuring whether a model can pass standards, solve complex mathematical problems, or pass many other benchmarks commonly used to determine the state of the art. There is no way to do it yet. Well, they can accomplish important tasks for us as journalists. Northwestern University’s Sachita Nishal, Charlotte Lee, and Nick Diakopoulos highlighted this challenge in a paper published earlier this year, which concludes with further research and industry recommendations. We are calling for your cooperation. On a smaller, project-by-project level, tools like Braintrust can also be helpful. Small-scale, domain-specific models: Many of us have wondered what would happen if we trained large-scale language models on news media archives. The problem is that a single archive is just a drop in the bucket compared to the internet-scale datasets used to train basic language models. Here you can borrow from their work in medicine, law, finance, and create small-scale, domain-specific models. Some of them are showing promising results. Research shows that smaller, domain-specific models can outperform larger underlying models on certain benchmarks, and enterprises are starting to take notice. What does that mean for us? Explainability and interpretability: Particularly in our profession, we find it difficult to trust the output of a model when even its creator cannot fully explain how the model works. It’s difficult to do. Model explainability, particularly in the area of neural networks, has been the subject of active research for some time (including a notable contribution by New York Times alum Shan Carter). Being able to understand and explain how these models work, even at the most basic level, will help your audience understand them. Andrei Karpathy’s research helped The Times achieve a similar outcome in 2023. Metadata generation: A number of interesting studies have shown that in certain situations, large language models can match or exceed human performance on complex classification tasks (for example, trying to assign There are. topics, sentiment, structural attributes of news articles, etc. When you can trust your language model to provide fairly accurate metadata about your journalism, it opens up a wealth of opportunities to analyze your articles and enhance your product offerings in a variety of ways. Maybe you want to extract all the evergreen stories from your archives, or see if explainers perform better than traditional story formats in certain situations. Companies like Overtone and SmartOcto are already doing some of this work, but news organizations could develop their own taxonomies to meet their own needs.
Our team is keen to collaborate in these and other areas. Please feel free to contact us.
Chase Davis is the director of the Minnesota Star Tribune’s AI Lab.
