Graph of Thoughts
Network-based reasoning where thoughts branch, merge, and loop — modelling how ideas actually connect and combine.
Introduced: Graph of Thoughts (GoT) was published in 2023 by Besta et al. at ETH Zurich. The paper demonstrated that modelling LLM reasoning as an arbitrary graph — rather than a chain or tree — enables operations like thought merging, looping, and refinement that significantly improve performance on tasks such as sorting, set operations, and keyword counting.
Modern LLM Status: GoT remains a research-stage framework that is not natively supported by current LLMs. Modern models still generate linear token sequences and do not spontaneously form graph-structured reasoning. Implementing GoT requires explicit orchestration — either through multi-prompt workflows or dedicated software that manages the thought graph externally. The core insight (that merging and looping improve reasoning) has influenced newer agentic architectures.
Beyond Linear and Tree Structures
Chain-of-Thought forces reasoning into a single path. Tree of Thought lets it branch. Graph of Thoughts removes all structural constraints — thoughts can branch, merge back together, form feedback loops, and create complex networks that mirror how humans actually solve hard problems.
The key breakthrough is aggregation: the ability to take multiple independent reasoning paths and combine their best elements into a single, stronger result. A tree can explore alternatives, but it cannot synthesise them. A graph can split a problem four ways, solve each piece independently, and then merge the solutions into a unified answer that none of the individual branches could have reached alone.
This matters because many real-world problems are not linear. Policy decisions balance competing priorities. Design work integrates constraints from multiple stakeholders. Research synthesises findings across studies. GoT gives LLM reasoning a topology that matches the structure of the problem itself.
GoT introduces three fundamental operations — Generate, Aggregate, and Refine — that transform a collection of thoughts into a directed graph. This enables reasoning patterns impossible in chains or trees: convergence, cycles, and multi-path synthesis.
Chain: A → B → C → D (linear, no branching)
Tree: A branches to B and C; B branches to D and E (branching, no merging)
Graph: A branches to B and C; B and C merge into D; D loops back to refine B (branching + merging + loops)
The graph subsumes both chain and tree as special cases — every chain is a graph, every tree is a graph, but not every graph is a tree.
Why It Works
Non-Linear Exploration
Unlike chains or trees, GoT explores multiple reasoning paths simultaneously without constraining them to a single sequence. Parallel branches investigate different angles of the problem at the same time, maximizing coverage within the same number of reasoning steps.
Thought Aggregation
The defining operation that separates GoT from all prior frameworks. Multiple independent reasoning branches can be merged into a single, stronger conclusion — combining partial solutions that no individual branch could have reached alone.
Backtracking
Unpromising reasoning paths can be abandoned without losing progress on other branches. The graph structure preserves all viable paths while pruning dead ends, ensuring computational resources focus on the most productive lines of reasoning.
The GoT Pipeline
A five-step orchestration that builds, explores, and refines a graph of interconnected thoughts.
Define the Problem Graph
Decompose the problem into an initial set of thought nodes. Identify which parts can be solved independently (parallel branches) and which depend on others (sequential edges). This graph topology is the blueprint for the entire reasoning process.
Generate Thoughts
Expand the graph by producing new thought nodes from existing ones. Each generation step can create one or many new nodes — exploring alternatives, breaking down sub-problems, or proposing candidate solutions for individual branches.
Explore and Score
Evaluate thought nodes using a scoring function. Promising branches receive more computational resources while weak branches are pruned. This keeps the graph focused on high-quality reasoning paths without exhaustively exploring every possibility.
Aggregate Thoughts
Merge multiple thought branches into unified results. This is the operation that distinguishes GoT from all prior frameworks — taking the best elements from parallel explorations and synthesising them into a single, stronger conclusion that no individual branch could reach.
Refine Through Loops
Feed aggregated results back into the graph for further improvement. Refinement loops let the system iteratively polish its reasoning — catching errors, filling gaps, and strengthening weak arguments until quality thresholds are met or improvement plateaus.
See the Difference
Chain or Tree Approach
Chain: Analyse economic impact → then environmental impact → then social impact → conclude. Each step only sees the previous one.
Tree: Branch into economic, environmental, and social analyses — but each branch concludes independently. No cross-pollination of insights.
The economic analysis cannot incorporate environmental findings, and the final conclusion is just a list of separate perspectives rather than an integrated assessment.
Graph of Thoughts Approach
Branch: Explore economic, environmental, and social dimensions in parallel.
Aggregate: Merge findings — economic analysis accounts for environmental benefits, social analysis incorporates economic data.
Refine: Loop back to strengthen weak areas identified during aggregation. The integrated assessment reveals trade-offs invisible to any single branch.
GoT in Action
“Sort this list of 128 numbers in ascending order.”
A single LLM call frequently makes errors on long lists — swapping adjacent elements, dropping items, or losing track of position.
Split (Generate): Divide the 128-element list into 4 sub-lists of 32 elements each — creating 4 parallel thought branches.
Sort (Generate): Sort each 32-element sub-list independently. Smaller lists are within the model’s reliable sorting range.
Merge (Aggregate): Pairwise-merge the 4 sorted sub-lists into 2 merged lists, then merge those 2 into the final sorted output.
Verify (Refine): Check the final list for ordering violations and fix any that remain.
“Analyse the impact of migrating our on-premise infrastructure to a cloud-based platform.”
Linear analysis tends to focus on one dimension at a time, missing critical interactions between factors.
Branch (Generate): Create parallel analysis nodes for technical feasibility, user experience, security posture, operational costs, and market positioning.
Cross-Link (Generate): Each branch generates findings that reference other branches — e.g., security improvements feed back into market positioning estimates.
Synthesise (Aggregate): Merge all five analyses into a unified recommendation that weighs trade-offs: “Upfront migration costs rise 12% but scalability gains reduce infrastructure spend by 30%, netting a positive ROI within 18 months.”
Stress-Test (Refine): Loop back through the graph with adversarial assumptions — what if productivity drops 15% instead of 5%? Update the recommendation accordingly.
“Design a solution for reducing food waste in urban restaurants.”
Creative problems benefit from exploring diverse angles and then combining the best elements — something trees cannot do.
Diverge (Generate): Explore 4 independent solution spaces: technology (smart inventory), behavioural (portion sizing), economic (dynamic pricing), and community (surplus redistribution).
Develop (Generate): Each branch produces 2-3 concrete proposals with implementation details, costs, and expected impact.
Converge (Aggregate): Merge the strongest elements into a hybrid solution: smart inventory tracking triggers dynamic pricing for items approaching expiry, while unsold items are automatically routed to a community redistribution network.
Iterate (Refine): Loop back to identify gaps — the hybrid solution needs a mobile app component (from the technology branch) to coordinate redistribution logistics.
Perfect For
Problems that break into independent sub-tasks whose results need to be combined — sorting, summarisation, and multi-criteria analysis.
Where insights from one domain should inform conclusions in another, requiring cross-pollination between reasoning branches.
Creative work that benefits from wide exploration followed by convergent synthesis into a unified output.
Complex decisions requiring iterative refinement after an initial round of analysis, where feedback loops strengthen the conclusion.
Skip It When
Simple question-answer or step-by-step procedures that do not benefit from graph structure.
GoT requires multiple LLM calls for branching, aggregating, and refining — not ideal when latency or cost is constrained.
Without software to manage the graph, manual multi-prompt workflows become error-prone at scale.
Use Cases
Where Graph of Thoughts delivers the most value
Large-Scale Sorting
Split long lists into manageable chunks, sort each independently, and merge results pairwise — the original GoT benchmark that outperformed Chain-of-Thought.
Multi-Criteria Decisions
Evaluate options across independent dimensions (cost, risk, impact, feasibility), then aggregate into a weighted recommendation accounting for trade-offs.
Document Summarization
Summarize sections independently, then merge section summaries into a coherent executive summary preserving relationships between topics.
Research Synthesis
Analyze multiple papers or data sources in parallel branches, then aggregate findings into a unified review identifying agreements, contradictions, and gaps.
Product Design
Explore user needs, technical constraints, and business goals in separate branches, then converge into a design specification balancing all dimensions.
Set Operations
Compute intersections, unions, and differences across large keyword sets by partitioning across parallel branches and merging partial results.
Where Graph of Thoughts Fits
The evolution from linear reasoning to graph-based networks
While Chain-of-Thought follows one path and Tree of Thought branches without merging, Graph of Thoughts is the first framework that lets reasoning paths converge — combining insights from multiple independent explorations into unified conclusions. This makes it uniquely suited for problems where synthesis matters more than selection.
Related Techniques & Frameworks
Think in Graphs
Build graph-structured reasoning into your prompts with our interactive tools, or explore more advanced reasoning frameworks.