Reasoning models aren't really reasoners, its basically neural style transfer protocol where you force a model "decoder" to emit tokens in a style that appears to be Reasoning like a deductive thinking.
I'm still amazed by how intelligent the outcome is, after these number crunching processes. Really cannot relate its ability to generalize information to the theory behind it.
Using LLMs as Optimizers: A new population-based method called LEO that leverages large language models for numerical optimization tasks like nozzle shape and windfarm layout design. Shows comparable results to traditional methods while highlighting unique challenges of LLM-based optimization
classic NN takes a vector of data through layers to make a prediction. Backprop adjusts network weights till predictions are right. These network weights form a vector, and training changes this vector till it hits values that mean "trained network".
Neural ODE reframes this: instead of focusing on the weights, focus on how they change. It sees training as finding a path from untrained to trained state. At each step, it uses ODE solvers to compute the next state, continuing for N steps till it reaches values matching training data. This gives you the solution for the trained network.
AI for science is much bigger than RL or Generative AI in science.
There are several classes of models Like operator learning, physics informed neural networks, Fourier operators
That perform magnificently well and have killer applications in various industrial settings
Do read the attached paper if you're curious about AI in science