May 29, 2009
How do you access all those repositories of knowledge that you use in Wolfram|Alpha?
It’s been lots of work. I mean we had a foundation which was, in terms of the algorithmic side of things, had a system called Mathematica which we’ve been building and selling out there in the world for 23 years now. And that’s the platform from which Wolfram|Alpha is built.
In terms of the actual raw data about the real world, so to speak, yes, we’re going to lots and lots of different sources, many of them are kind of real-time data feeds and so on that come into our computer systems. The main thing we’ve been able to do is we built a kind of a pipeline, a kind of an almost-industrial process for taking outside data from different sources and being able to organise it, automatically validate it, use statistical methods and visualisations and so on to see which of these pieces of data are likely to be questionable and so on. And then kind of putting human experts into the pipeline, letting people who really know different domains of data tell us things about what’s reliable, what’s not, how one should combine things and so on.
I mean this is a lot of work and there’s a lot more work that we need to do along these lines of ingesting all these different sources of data and setting them up so that they can really be… they’re really organized enough to be computable with.