What are some advanced concepts in programming that most average programmers have never heard of? originally appeared on Quora: the place to gain and share knowledge, empowering people to learn from others and better understand the world.
Answer by Tikhon Jelvis, studied programming languages and did research on program synthesis, on Quora:
Probabilistic Programming
Okay, I have to admit, I live in something of a programming language filter bubble—among people I know, probabilistic programming languages are a regular topic of conversation. So I was quite surprised to learn that, outside my bubble, most people haven’t even heard of the idea. This includes programmers, data scientists and people specializing in probability and statistics.
Probabilistic programming languages are an interesting meeting point of several different fields, a mix of:
- Programming languages—think of it as a new programming paradigm
- AI and machine learning
- Probability and statistics
There are two ways to view probabilistic programming language. One way is that it’s a language with native constructs for sampling distributions, letting you efficiently evaluate programs with random behavior. Another way is that they are a rich way to specify complicated probability distributions, more general than the traditional techniques used by mathematicians and scientists.
Either way, it’s a new technology that unlocks legitimately exciting possibilities for programmers and scientists. People who don’t know about these languages are really missing out.
It’s a bit hard to explain exactly what probabilistic languages accomplish. The problem isn’t that the idea is too complicated but that it’s too simple—it’s just a language with a sampling mechanism built in! How is this different from just using Math.random() in JavaScript? And, in some sense, this is a probabilistic language, just one that isn’t very interesting. What makes dedicated probabilistic languages special is that they let us solve problems that would be intractable otherwise.
My favorite example of this in action is the “tug of war” demo I originally saw in a talk by Noah Goodman. Turns out the example comes from his book Probabilistic Models of Cognition which, happily, is available online. (See: ProbMods: Conditioning)
I won’t cover all the details here. The basic idea is that we can write a simple model of tug of war—each player has some unknown level of strength, and teams of players compete with the total strengths of each player determining which team wins. The code to express this is short (< 20 lines) and direct: about the same code you would write to just simulate this simple model. Doing this in a probabilistic language, though, gives us something that would be really expensive with a straight simulation: we can determine the probability of a team winning conditioned on previous results involving those players even in different teams with players not in the current match.
This really short model, evaluated with a probabilistic language, gives us a really powerful algorithm for measuring the skill of individual players based on their performance in team-based games. Think of it as a version of ELO generalized to team games—basically Microsoft’s TrueSkill algorithm. With probabilistic programming an algorithm that led to a pretty popular paper becomes a 20 line example used in introductory talks.
This question originally appeared on Quora - the place to gain and share knowledge, empowering people to learn from others and better understand the world. You can follow Quora on Twitter, Facebook, and Google+. More questions:
