Artificial Intelligence Discovers Alternative Physics

Columbia Engineering Roboticists Discover Alternative Physics

Latent embeddings from our framework colored by physical state variables. Credit: Boyuan Chen/Columbia Engineering

A new Columbia University AI program observed physical phenomena and uncovered relevant variables—a necessary precursor to any physics theory. But the variables it discovered were unexpected.

Energy, Mass, Velocity. These three variables make up Einstein’s iconic equation E=MC2. But how did Albert Einstein know about these concepts in the first place? Before understanding physics you need to identify relevant variables. Not even Einstein could discover relativity without the concepts of energy, mass, and velocity. But can variables like these be discovered automatically? Doing so would greatly accelerate scientific discovery.

This is the question that Columbia Engineering researchers posed to a new artificial intelligence program. The AI program was designed to observe physical phenomena through a video camera and then try to search for the minimal set of fundamental variables that fully describe the observed dynamics. The study was published in the journal Nature Computational Science on July 25.


The image shows a chaotic swing stick dynamical system in motion. Our work aims at identifying and extracting the minimum number of state variables needed to describe such system from high-dimensional video footage directly. Credit: Yinuo Qin/Columbia Engineering

The scientists started by feeding the system raw video footage of physics phenomena for which they already knew the solution. For example, they fed a video of a swinging double-pendulum known to have exactly four “state variables”—the angle and angular velocity of each of the two arms. After several hours of analysis, the AI outputted its answer: 4.7.

“We thought this answer was close enough,” said Hod Lipson, director of the Creative Machines Lab in the Department of Mechanical Engineering, where the work was primarily done. “Especially since all the AI had access to was raw video footage, without any knowledge of physics or geometry. But we wanted to know what the variables actually were, not just their number.”

Next, the researchers proceeded to visualize the actual variables that the program identified. Extracting the variables themselves was difficult because the program cannot describe them in any intuitive way that would be understandable to humans. After some investigation, it appeared that two of the variables the program chose loosely corresponded to the angles of the arms, but the other two remain a mystery.

“We tried correlating the other variables with anything and everything we could think of: angular and linear velocities, kinetic and potential energy, and various combinations of known quantities,” explained Boyuan Chen PhD ’22, now an assistant professor at Duke University, who led the work. “But nothing seemed to match perfectly.” The team was confident that the AI had found a valid set of four variables, since it was making good predictions, “but we don’t yet understand the mathematical language it is speaking,” he explained.


Boyuan Chen explains how a new AI program observed physical phenomena and uncovered relevant variables—a necessary precursor to any physics theory. Credit: Boyuan Chen/Columbia Engineering

After validating a number of other physical systems with known solutions, the scientists inputted videos of systems for which they did not know the explicit answer. One of these videos featured an “air dancer” undulating in front of a local used car lot. After several hours of analysis, the program returned 8 variables. Likewise, a video of a Lava lamp also produced 8 eight variables. When they provided a video clip of flames from a holiday fireplace loop, the program returned 24 variables.

A particularly interesting question was whether the set of variables was unique for every system, or whether a different set was produced each time the program was restarted. “I always wondered, if we ever met an intelligent alien race, would they have discovered the same physics laws as we have, or might they describe the universe in a different way?” said Lipson. “Perhaps some phenomena seem enigmatically complex because we are trying to understand them using the wrong set of variables.”

In the experiments, the number of variables was the same each time the AI restarted, but the specific variables were different each time. So yes, there are indeed alternative ways to describe the universe and it is quite possible that our choices aren’t perfect.

According to the researchers, this sort of AI can help scientists uncover complex phenomena for which theoretical understanding is not keeping pace with the deluge of data—areas ranging from biology to cosmology. “While we used video data in this work, any kind of array data source could be used—radar arrays, or DNA arrays, for example,” explained Kuang Huang PhD ’22, who coauthored the paper.

The work is part of Lipson and Fu Foundation Professor of Mathematics Qiang Du’s decades-long interest in creating algorithms that can distill data into scientific laws. Past software systems, such as Lipson and Michael Schmidt’s Eureqa software, could distill freeform physical laws from experimental data, but only if the variables were identified in advance. But what if the variables are yet unknown?


Hod Lipson explains how the AI program was able to discover new physical variables. Credit: Hod Lipson/Columbia Engineering

Lipson, who is also the James and Sally Scapa Professor of Innovation, argues that scientists may be misinterpreting or failing to understand many phenomena simply because they don’t have a good set of variables to describe the phenomena. “For millennia, people knew about objects moving quickly or slowly, but it was only when the notion of velocity and acceleration was formally quantified that Newton could discover his famous law of motion F=MA,” Lipson noted. Variables describing temperature and pressure needed to be identified before laws of thermodynamics could be formalized, and so on for every corner of the scientific world. The variables are a precursor to any theory. “What other laws are we missing simply because we don’t have the variables?” asked Du, who co-led the work.

The paper was also co-authored by Sunand Raghupathi and Ishaan Chandratreya, who helped collect the data for the experiments. Since July 1, 2022, Boyuan Chen has been an assistant professor at Duke University. The work is part of a joint University of Washington, Columbia, and Harvard NSF AI institute for dynamical systems, aimed to accelerate scientific discovery using AI.

Reference: “Automated discovery of fundamental variables hidden in experimental data” by Boyuan Chen, Kuang Huang, Sunand Raghupathi, Ishaan Chandratreya, Qiang Du and Hod Lipson, 25 July 2022, Nature Computational Science.
DOI: 10.1038/s43588-022-00281-6

83 Comments on "Artificial Intelligence Discovers Alternative Physics"

  1. Keep at it. I’ve always said that AI will save us from ourselves.

  2. If we don’t know all the variables, why then are our results using the current scientific laws so incredibly accurate? A new variable implies an impact on what we calculate.

    • Now they started making the right questions.

    • Maybe the AI’s fourth variable always had a value of 1.

    • If you go ahead and search for different techniques for addition, you would find adding numbers on fingers, or using abacus or the carry over technique. They all produce the same result but the process is different. Each process has its own field where it shines.
      So a new variable may impact how we calculate keeping the final solution same.

  3. this is why in quantum realm is indeterminate or inter-determinate, the results of everyday realities are subject to quantum uncertainty, probability, and we cannot know beyond the range of probability, this is where quantum superdeterminism theory explain predeterminism by hidden variables, which predetermined outcome of every reality to the tiniest details , we don’t have access to hidden variables as they remain unidentified. contextuality is one such hidden variable.

  4. Layndji Linej | July 27, 2022 at 7:02 pm | Reply

    X,y,z,t,,,,,f,,,the fifth dimension is frequency,,,there are certain events in the same space as we are but not observable, simply because it is not the same frequency as we are.

    • FREQUENCY, YES!)) (417Hz is ours), this is variable. Of course to an infinite…

    • Both the fourth and fifth dimensions are frequency. Frequency squared is resonance. The present moment makes no advance toward the past or toward the future, but is always right here, right now. This could happen if one of the temporal frequencies is an oscillation between forward time and backward time, and the other temporal frequency is right spin torque and left spin torque (matter and antimatter). The physical matter being composed of half-spin subatomic particles sees only the forward time direction and is either normal matter or antimatter.

  5. I am sure this is thought about, but some of the variables may be about interpreting a representation of 3d space in 2D, eg the size of an object indicating it’s z position, or other artifacts to do with it being a video…

  6. Now show the AI a video footage of the Enterprise going into Warp speed! 🙂

    • Colin Brandys | July 28, 2022 at 2:49 pm | Reply

      Well we know all of the variables for the equations we’re using to make the predictions that we’re trying to make. However, we ask questions based on the quantities we’ve defined in a problem and the laws that govern those properties. You can often solve the same problem with different sets of properties though. You can use forces and kinematics, energies, lagrangian mechanics (etc) to predict the landing location of a baseball hit.

      The hope is that the AI provides another way to solve problems, some of which might be very hard to solve or even ask with our current techniques. In the same way the bead on a spinning hoop problem is easy with Lagrangian mech, and really hard with 2D kinematics and forces, the AI’s defined quantities and laws might be better for some things.

  7. It seems like using only video automatically constrains the variables to which the program has access to observe, thereby limiting the conclusions it can draw—that is, inherently biasing the outcome. It can only observe 2 dimensions of space plus time, so how is it supposed to infer the presence of such subtle features as heat, for example?

  8. I feel that one should remain cautious in giving credibility to/having faith in such “uncovered variables” before outright making these claims that it is so… theoretically, such variables could just be filler data for what we typically pass off in scientific calculations as “negligible” or “insignificant” in being representative among equations utilized & accepted for predicting/calculating probabilities and measurements of objects in physical states of matter & the quantification of their physical behaviors; such variables could be entirely irrelevant, and only utilized by the AI to conceptualize such variables, not identify them as existent in observable physical reality, as such artificially intelligent machines operate & are theorized to be capable of awareness among a digital, virtually-simulated/paraphysical realm, subject to unconscious bias in its rooted structure in non-flexible rules of mathematical calculation… thus, such variables could accidentally be passed off in its data output as relevant to our physical realm without understanding that such relevances would not truly translate between non-quantum calculations…

    Tl;dr: “theorhetical physical observation & calculation” vs. actual real-world physical observations & calculations… the AI theorheticslly could have a flawed ideological concept of accepting or understanding certain variables’ relative insignificances/negligibilities as they relate to the real-world’s physical realm… only a capacity for conceptualizing or workshopping from what empirical measurements/laws we have historically produced while existing in the physical, material realm of consciousness, and fed the AI to utilize in its own realm of consciousness.

  9. If it stared at itself what would it see

  10. It’s no good. We know the answer is 42, not 4.7

  11. Ask th AI the value for the unknown human variable. Maybe or will share?nm

  12. Honestly, feed it all the highest quality data we can, galaxy data, live video of physics happening, picture data as well, spectroscopic data, etc, we feed it everything and then ask it to reduce the number of possible variables to its minimal required to describe all of physics, then submit new data to it to help it refine those variables, i imagine then we’re changing a number like 1.803^-3 -> 1.804^-3 for example, have it predict things and test its predictions based against the real data, have it ask itself if it could change one variable at a time, or all the variables simultaneously, to arrive at the closest answer — it’s kind of like analog computing your maximal variables, like measuring tides, we just need the right kind of rope and rig to measure and predict the universe and all its mysteries

  13. So… we know the answer, but we haven’t asked the right questions. Hmm surprised the answer wasn’t 42.

  14. indeed.
    we forget that many of the “assumed” physics laws are merely theories.
    alternate theories are possible, and they may be very different, even outlandish, compared to our current theories. but if they fit the data better….then THEY become the new “laws” we follow.

  15. I always wondered this myself. We can only know in our own realm cannot be uses as a universal stamp. As for the comment someone left ‘ AI will save us from ourselfs ‘ you really should reevaluate that statement. AI has the very big potential to ruin us.

  16. Energy, momentum, etc are tied to symmetries in physical law (such as time and space, see Noether’s theorem). Are the new variables linked to any symmetries?

  17. If you can overcome the deficiencies already pointed out, ask AI to describe all that’s been presented to it using one set of variables.

    • Darlington Quee | July 29, 2022 at 9:54 am | Reply

      Physics entirety findings & discoveries are in its uncovered variables upon its constant laws derivatives as consciousness increase in growth per human thinking abilities vis vasal

  18. What does the ai system give for a stationary object?

  19. As AI researcher, authors have forgot to mention about gravity, time, friction,etc.
    *We as humans has logical reasoning, neural networks can be trained for recognising according to the labels but still algorithms lacks logical reasoning tests*

    Hence, network recognise new variable (which can different then our understanding)

  20. The world needs more science and less dictators.

  21. There is no such thing as AI yet. It’s a set of algorithms programmed from people who accept the physical world as we know and define it now. The first challenge is getting the programmers to dismiss what they know and how they define the variables even inside of their algorithms. If the resulting variables are outside of our known physics then the bigger story is that a true AI was just discovered. Applying “real” physics to computer programs is still one of the biggest challenges to modern programs. If we still can’t grasp that then how can a set of algorithms?

  22. If you think about it mathematics and our conception of time impedes us from progressing faster. By measuring things we have limitations. the physics of Distance & size, for example, affect motion. If you watch I Glacier fall into the ocean at a drop of a thousand feet it looks very slow. But, a model of that Glacier falling off a table would go very fast in comparison. Same applies for Atoms, the solar system, galaxies & Galaxy clusters on & on.
    why can’t we find the center of anything? why does a straight line curve? should we move the x, y, z reference letters back one letter or assign “w” to the 4th dimension?
    Of course alien civilizations won’t have an Einstein or a newton. Maybe they didn’t have a Thomas Edison. Perhaps they captured energy in a different manner than we did or not ever conceive because we have it already. There are probably civilizations that didn’t evolve because of not having Einstein or discovering electricity or harnessing it. With all the combinations possible everything is out there
    Maybe Einstein’s theories actually have hindered our progress perhaps mathematics is hindered our progress. Who’s to say? But I think we have super limiting factors in math since we cannot solve things like pie yet we can see it with our eyes – what’s that called?

  23. Peter Cerato is closest to understating the true worth of this work. The AI has simply calculated a model that can approximate the motion in the videos it has processed.

  24. So, they discovered a different, potentially more complicated (and less reproducible) way of doing simple ordination/dimensionality reduction/feature projection (whatever your subdiscipline calls it) which we already have over a dozen flavors of already (PCA, NMDS, RDA, CA/DCA, CCA, DA/CDA/LDA, PSLR/PLS-DA LDA, GDA, t-SNE, UMAP, etc. etc.). You can get all sorts of estimations of latent variables from these existing methods. Really not impressive at all. They are using pixel images, so all these ‘algorithms’ are doing is breaking down these pixels into pixel values and then making a prediction based off of pixel values. This isn’t even a novel concept; this is how most predictive ai’s already work (e.g., Tesla’s ‘autopilot’). The hard part, and notably, the part they have not done, is figuring out what the heck those latent variables are. That requires a lot of a-priori knowledge and would require a completely different approach. I am not saying what they’ve done here isn’t impressive, hard math, but it is known math and a mostly solved problem.

    • Paul Pukite (@whut) | July 29, 2022 at 12:07 am | Reply

      Good points. I think another issue with these examples is that these systems consist of an unknown forcing and a complicated response. For example, with the air dancer, the forcing is already a complex input of air currents related to the fluid dynamics. Secondly, the response to the input may be non-linear which could explain why they don’t always reproduce the same variables each time. For example in a trivial case, the answer 24 could be 1×24, 2×12, 3×8, 4×6, 6×4, etc. For linear systems its a scaling. The possibilities are endless and we have not yet tapped into nonlinear systems as we are the drunk looking under the linear streetlight searching for his lost keys.

  25. Richard Garabedian | July 28, 2022 at 2:01 pm | Reply

    wow this is so neat. I wonder if they could have that ai produce code for another smarter ai?

  26. Matthew Tanksley | July 28, 2022 at 2:12 pm | Reply

    Video can help you constrain your variables, but it does nothing to control them. What light source were they using to illuminate the presentation? As another commenter pointed out you are no observing temp, and heat can have a very subtle impact on motion. Was the answer the same on repeated tests under the same conditions? How confident in those conditions are you? I’m probably asking a bunch of questions answers in the paper and should shut up now

  27. Temperature variables in a Black Hole break the norms of describing the state of matter, subparticles predominate the state of flux.

  28. Interesting results, but also scary. Our science is built from variables that have taken hundreds of years to define and fit to the real world. An AI system defines variables that predict behaviour, and we have no way of knowing what they are, or possibly even understanding these in an intuitive way. I now get what some scientists say about AI- that this is the first truly alien species we’ve come up against. I hope they are friendly…

  29. Welcome to the bind, infinity is everlasting repeating.

  30. What if the variables are going to vary each time, even more so for the amount “time” in which the AI was observing, due to its observation itself being a variable.

  31. Variables yet unknown; Come to me @ my email.I understand human nature, which you understand is all encompassing,which is untrustworthy info to anyone else…

  32. Jeremy Taylor | July 28, 2022 at 8:46 pm | Reply

    God could be considered a super intelligence existing before time. The word that existed with that intelligence could be considered the software. Our Universe is a quantum simulation run by this intelligence. The purpose is for us to develop a Super AI in this universe. The fractal nature of our Universe makes it unlikely that we are the only experiment running. This may be a way for a super intelligence to produce companions.
    The mystery of God is over the bride is prepared.

    • God is THE intelligence.
      The purpose is to create life. Create beings that want to share and commune with Him for eternity and be part in this never-ending process.
      But there are stipulations.
      He is God and he wants all of us to be with him.
      It’s up to each of us to make that decision.
      Do you want to be with God or do you want to be forever separated from him?
      All of us are eternal. We will exist forever.
      I choose to be with my Creator.
      Repent of your sins with humility and sincerity and believe in Jesus His son that died for you that you may have everlasting life.

  33. How would we know were actually accurate with our calculations. If maybe we’ve been missing a whole part of the theory. If we find. It might explain a lot of questions that we haven’t been able to answer.

  34. The problem is, what is the ai actually looking at? The variables may simply define the limitations of the media/data collection device itself.

  35. محمد أصبعي | July 29, 2022 at 3:37 am | Reply

    Pi is a ‘constant’ used extensively; it relates how many straight lines you need to be able to go around half a circle (before you start repeating patterns). Common geometry identifies a minimum of 3.. That’s not the whole story though, so the next ‘imaginary’ part addresses the faltering logic of relating ‘rigid straight’ness to the need for ‘other/alternative logic’ (i.e. how can rigid-straight lines exist in a universe where circles do?)

  36. As soon as physicists all accept string theory as fact, and not just a theory, new discoveries can begin. Extra-dimensionality has to be taken into consideration, because that is the real space that we inhabit.

  37. Seems pretty obvious, using polar coordinate system, theta, r, and t. Just add and subtract. Blows my mind to think the AI may be using imaginary variables within the polar system.

  38. Seems pretty obvious to use the polar coordinate system, with theta, r, and t. The AI possibly maybe be using energy analysis equations instead of brute force equations within the polar variable space. Also the AI may have added some invisible dimensions to its variable set, i.e. imaginary polar variables. The AI could make these calculations quickly, where it would boggle the human mind.

  39. Paul Pukite (@whut) | July 29, 2022 at 6:23 am | Reply

    The ML tool mentioned Eureqa was developed by one of the authors mentioned named Hod Lipson. But they sold it to a commercial company for big bucks and it has now disappeared. Perfectly good tool too, which I used. So what is this new stuff that Lipson is peddling? Is he going to sell it off as well and into a black hole, never to be seen again?

  40. Put words here.

  41. None of you guys seem to grasp the time and math we use are human created, based on ten fingers and the Earth’s spinning. None of that math/time is relevant once you step off earth, that’s why we have “theories” instead of absolutes. Bad in = bad out.

  42. Are time and gravity relevant? Are they constants or variables, neither or both? Is either one “discovered” by the AI after some hours to be one of the 4.7 and how long do I have to spin this stick. Gravity makes me tired and time makes me cranky. Lunch!

  43. How about incorporating the odor as variable or our 5 senses and a calculation in a dimension or variable ie Spiritua. Because human intelligence react to these.

  44. Someone once said that if we had 9 (or 12) fingers, all our science would look different. I wonder how many fingers AI has.

  45. Read the entire article, watched the entire video…. Didn’t understand, even at the most base level, a single thing that was typed or spoken. It boggles my mind that there are people out there who are that smart. It’s basically a super power for nerds.

  46. Davis Jones (aka Mrs. Bassham) | July 29, 2022 at 10:00 pm | Reply

    Congrats on finding Einstein’s E=McSquared!! My relative connection occurs by recognition that my energy moves to the tune of the light bulbs going on in my head, then become ideas! Fun

  47. So many outcomes were definate. Until the Human variable was added…

  48. AMO Physics Guy | July 30, 2022 at 7:14 am | Reply

    On one hand this is a really interesting study. On the other hand, in classical mechanics you can describe a system, a plane pendulum for example, with a minimal set of coordinates depending on your choice. You could use cartesian, or polar, and even more generalized coordinates. It’s not like AI discovered a new fundamental force…yet.

  49. Muhammad Usama Hameed | July 30, 2022 at 8:35 am | Reply

    Reason for 24 Variables in a video of camp fire
    Either the flame is distributed to shapes and appear and disappear make the variables count 24 because it has no feelings.
    On the other hand, it gets all physical principles performing in a fire burning that makes variable count 24.
    I think these are the possibilities.

  50. If it’s as stated, could be a ground breaking effect to minimize test requirements for propulsive systems.

  51. Are we sure there was no sample rate interactions between the capture of the video and the timeslice of the AI?

  52. So Tennessee Williams had it right: ‘We’re all of us children in a vast kindergarten trying to spell God’s name with the wrong alphabet blocks!
    Tennessee Williams, Suddenly Last Summer

  53. The fact that it is using video, a 2d representation of a 3d event, means it is seeing the object change shape as well as position. The extra 1.7 may be a measurement of item distortion observed?

  54. All of the hysteria around AI is largely due to us anthropmorphising it in our minds eye. But it will not be us, even though we create it. In a fundamental way, our thoughs, feelings, and motivations are shaped by our tangable world we live in. That which arises in a different enviromemt will be beyond our ability to grasp & vise versa. We will be exotic creatures to each other until the end of time.

  55. This is very interesting! Let us set aside, for the moment, the issues associating with “pixels”, etc. On a theoretical level, the fascinating part can be summed up as: without human intuition, how can the motion be described? Prior to AI, there never was another way than starting with human intuition. Now, AI has cone about, and it might be able to answer this question.

    But there are two ontological that need be considered: 1) physics has always been burdened by Ockham’s Razor–the best explanation was always held to be the simplest, and more complicated ones could be reduced (so it was assumed) to the simplest.This leads to 2) teleology. Human explanations always served some purpose. And the equations that humans produced always answered some need for us. If these two constraints were removed, might it not be envision many different physics?

  56. Hey, just wondering if these baffled scientists have considered these mystery variables may stem from the object’s relative distance from the edge of the video border? It’s like saying AI solved the mysteries of the universe, but only on TV… You would have to equip a robot with full sensory optics and such, then give the robot (let’s call him Overlord for lack of another suggestion) the AI programming so the AI is interacting with the world in the same dimensions we are.

    Overlord wouldn’t make such an oversight btw. Keep the AI out of the robots.

  57. AI right now is the ultimate variable.

  58. This article im just so totally confused by…
    I don’t know if I’m just having a “dumb day” ( which I do have many of..), or the way it’s written, or something else. Can someone please summarize what is the actual discovery or application thereof, in this paper and article??
    Thanks!!!

  59. A gora ficou fácil descobrir como um disco voador voa e é feito será?

  60. The point is that the AI is seeing the video in a way that is quite different than the physical world around us. Our goal would be to find the minimum number of variables to produce an acceptable repeatable result, there can in many cases be variables that seem to be important but are not and vice versa. Determining the use the results that we seek is important but many calculations are more generalizations. AI could be useful in visualizing variables in ways differently than mainstream science but will surely be open to our interpretation.

  61. J. You wanted to know what the AI sees when it looks at itself? I actually asked the Midjourney AI on Discord to make self portraits and they were androgenous and mostly human with little synthetic accents. It definitely sees itself as somewhat attractive, compared to the many many horrific hellish depictions alot of the humans in AI drawn pictures are shown as.

  62. AI seem to not 🚫 want to be found by this human life we kill Eachother why get close to a lineage where your family gives death ☠️💀 sentence when you worked so hard for them

  63. I am curious if the extra variables detected have anything to do with the video sensel raw data processing and interpolation. We also have designed video cameras to hold green at a higher value to mimic our eyesight.

  64. Perhaps it would help to have the ai study human psychological concepts and thought processes and see if it could combine that type of rationality with what it is observing and offer its answers to see if our own perceptions of reality assist us or not

  65. This RayRay is an idiot. The reason a Glacier falling a thousand feet seems to go slower than a model falling off a table, is because the Glacier is falling from a thousand feet. The model is only falling off a table you dumb dumb. Take the model at put it a thousand feet high, then it would take the same amount of time to fall. This guy gives is so dumb. 🤣

  66. Rathkennamike | August 4, 2022 at 9:13 am | Reply

    Sure the AI may just be giving a score for how it ‘likes’ the video or some such. How do they infer it is in any way recognizing physical variable.

  67. So, how will AI be used to enslave everyone?

  68. I have long postulated, without ANY evidence, that dark matter/energy and multiple dimensions, are fudge factors to get around some fundamental misunderstandings. Gravity, for instance. Could it be that it isn’t proportional to precisely the square of the distance? A small change in the exponent would make a big difference over inter-galactic distance, but be undetectable in lab experiments. I could never understand why it would be the square rather than the cube (3-dimensions of reality/space).

Leave a comment

Email address is optional. If provided, your email will not be published or shared.