In-situ graph reasoning and knowledge expansion using Graph-PReFLexOR

Key points

• Problem: The author argues that much of current AI relies on models that produce outputs directly, without an explicit intermediate reasoning process akin to scientific thinking, and that LLMs often excel in fluency and pattern recognition yet typically lack structured reasoning. The goal is a model that goes beyond retrieving information to learn relations between concepts and abstract patterns and generalize across new domains.
• Input and output: This is a method paper that presents a training approach and a 3B model trained with it. Given a question or task, the model generates, inside a section wrapped in thinking tokens (<|thinking|>), a knowledge graph (concept nodes joined by relation edges such as IS-A, RELATES-TO and INFLUENCES), symbolic abstract patterns, reasoning steps and a hypothesis, and then gives the final answer.
• Method: Training data are generated on the fly during training: a passage drawn at random from the raw corpus becomes a question, RAG (retrieval-augmented generation) adds related content from the whole corpus, and the pipeline builds a correct answer containing a knowledge graph plus a rejected answer. In recursive reasoning, a critic agent critiques Graph-PReFLexOR's thinking section, the thinking is revised with that critique and the question is answered again, and after N rounds the last answer is taken or all answers are integrated. In the knowledge garden, Graph-PReFLexOR answers a next question that a human or another LLM agent builds from the nodes of earlier graphs, producing a new graph, and the graphs are joined into one growing graph.
• Implementation: The base model meta-llama/Llama-3.2-3B-Instruct is trained on data built from the raw text of around 1,000 papers on biological and bio-inspired materials, in the same order as PReFLexOR, ORPO followed by EXO, a DPO (Direct Preference Optimization) variant, and the critic in the recursive reasoning implementation is the non-fine-tuned Llama-3.2-3B-Instruct. The paper states that code and model weights are available on Hugging Face under lamm-mit and in the GitHub repository lamm-mit/PRefLexOR, and names the model lamm-mit/Graph-Preflexor_01062025. The GitHub README documents the PRefLexOR training package, ORPO and EXO trainer examples and Colab inference notebooks, says the model weights come from Hugging Face, and its top-level files include a Graph-PRefLexOR training notebook (Graph-PRefLexOR_train_Phase-I-and-II.ipynb).
• Results: On one task, designing a tough protein material that integrates a snowflake and ant behavior, GPT-4o as judge gave the first iteration Graph-PReFLexOR response an average of 9.7 and the reference model, a standard non-fine-tuned model, 6.4. Of the seven criteria, only Clarity and Organization favored the reference model, 9/10 against 8/10 for Graph-PReFLexOR. In the autonomous bio-inspired materials run, a human gave only the first prompt and an LLM wrote every later prompt, the graph grew over 12 iterations, and the nodes with the highest degree (number of linked edges) were Sustainable Materials, Self-healing and Adaptable Materials, and Ecosystems.
• Limits: The only comparison with a baseline model is Table 1, one task scored by a single judge model, GPT-4o, and the remaining results are case studies that show graphs and answers. The author notes that on the song task the model may be limited in capturing the temporal and emotional nuances essential to musical composition. Scaling to larger datasets and more complex models and interpretability of deeply interconnected graphs are named as future work, and the author writes that the critic could include fact-checking or even run simulations.

Paper links

External research summaries. These are not HDATF publications or measured product results.