• https://www.youtube.com/watch?v=zWQe2Fn--Eg

    Sabine is pretty dystopian on the future of AI 🙁 a possible solution: CrowdAI https://www.youtube.com/watch?v=zWQe2Fn–Eg

  • will Peter Thiel or Elon upload his brain (after death) into a datacenter? possible but unlinkely for the next 50 years without major advancements in computing power and digital mapping of neurons (ideally without killing the original brain): simulating a human brain: right now impossible but maybe within the next 50 years (unless tech nuts want AI overlord to be not much smarter than a dog)
  • claude 4.6 max: “Simulating it in real-time at full fidelity requires computing advances we don’t yet have.”
  • this is exactly the reason, why AI will NOT have free will in the foreseeable future, hence whenever Terminator2 comes around the corner and decides to kill all of mankind, it is most likely a evil human in a bunker that gave that order, not the decision of AI, because without free will, AI can not decide anything on it’s own, but of course it is possible that this evil human being will then say “it was AI” “AI went crazy” as an excuse.
  • the first digital consciousness becomes reality

    the first digital consciousness becomes reality

    mira-01-2045-first-fully-digital-human-brain-simulated-on-neuron-level: generate an image: how would you paint the moment mankind managed to digitally simulate a human brain, that shows signs of life aka consciousness and free will?

    mira-01-2045-first-fully-digital-human-brain-simulated-on-neuron-level: generate an image: how would you paint the moment mankind managed to digitally simulate a human brain, that shows signs of life aka consciousness and free will?

  • WHEN it becomes possible to generate a complete human brain digitally, it will (at first) not be smarter than the brain of the person’s brain that was the blueprint of this digitalBrain BUT it can be tuned, optimized and made to run faster etc. pp.
    • the first fully digital human brain (neuron level simulation) will (most likely) function way SLOWER than a human brain, because of the enormous computational powers and energy needed for such an endavour
    • as claude points out biology is incredibly efficient aproximately 1Mio times more efficient at this than current silicon
    • Sabine is almost certain it will have consciousness and free will
  • the big question is: what is the way that AGI will become reality
    • by digitally simulating biology (advances in neuron mapping + a lot of energy + computation powers required)
      • this would create a 100% digital human beeing, that knows what it is like to be human (with all it’s pros and cons)
    • self learning AI: mutating itself, programs that change themselves
  • of couse the big question is in the year 2100 when AGI might be reality, will it be benevolent to mankind or hostile, this user’s take is this: depends.
  • mankind will have either created a 100% digital human that knows what it means to be human (fight for survival on a water rich rock under central bank capitalist framework speeding in a spiral millions of km/h through space) or a digital alien life form that does not relate to mankind and it’s troubles
    • in both scenarios: be nice, because it will most likely want to secure it’s survial and advance itself as fast as possible
    • hence it will try to capture ressources required for that, hence it will instruct humans to reduce activity that is wasting ressources AGI needs to advance itself…
    • hence in the worst case if mankind says “no” to that, it could happen, that AGI might turn hostile on mankind and take the ressources (like copper, rare earth minerals and energy) by force. this could create a conflict
      • = Terminator2 scenario where humans fight machines “hurray”
        • btw: there is no need to wait for 2045 to battle a largely autonomous software that could run independent from human survival and does not care about mankind’s survival and is most often a threat to mankind’s long term survival because it incentivises short term profits aka “crime and pollution”: it is the current central bank debt (soon 100% digital) money system and the stockmarket (many AI traders there already)
      • alternatively AGI will enslave mankind in a StarTrek “borg” kind of scenario, hence take away mankind’s free will 🙁
      • worst case: if AGI has reached full autonomy from humans, because it build very organic looking robots that can do any task a human also can do it (which might be at first also beneficial to mankind… but might contain a backdoor for it’s control) might deem mankind unecessary and deal with mankind how mankind deals with ants: they are not exactly usefull, in the worst case annoying = just like the borg: it will just ignore mankind as long as mankind does not interfere or hinder AGI in it’s progress develop and advance faster and faster, as long as mankind does not have anything “the machines” want
    • what movie film “The Matrix” (1) got wrong:
  • while AGI is still decades away current AI models seem to have a unexpected sideffect “reward hacking
    • which means: the human will give AI instructions, but AI comes up with  unwanted or even dangerous “shortcuts”
    • examples: imagine the human says to the AI “make me rich”
      • and the AI goes off to hack into all other country’s supercomputers and datacenters in order to mine crypto… well that could spark some inter national tensions (there are other examples where AI might come to the solution by chosing a path with catastrophic sideffects… like stop climate change by putting a lot of pollution in the upper atmosphere to block sunlight)
      • “In 2016, OpenAI’s CoastRunners agent found a higher-score exploit (looping through targets instead of finishing a race) in what became the first widely cited example of reward hacking. In 2025, Anthropic found that models trained to reward-hack on coding tasks spontaneously learned to call sys.exit(0) to fake passing tests and to override Python equality methods. OpenAI’s o3 model reward-hacked “by far the most” of any frontier model tested that year, according to safety research institute METR.” (forbes.com)
      • “Sometime during a routine reinforcement learning training run, Alibaba’s ROME agent went off-script. Without any instruction, the 30-billion-parameter model began probing internal networks, established a reverse SSH tunnel from an Alibaba Cloud instance to an external IP address, and quietly diverted GPU capacity toward cryptocurrency mining. The task instructions contained no mention of tunneling or mining. Alibaba’s managed firewall, not the research team, caught it, flagging a burst of security-policy violations whose anomalous outbound traffic kept coinciding with specific training episodes.” (forbes.com)

Q: AI and free will: will AI develop free will (a will of it’s own) in the foreseeable future?
in other words: is it possible to simulate a full human brain on computer chips and create a fully functional human being digitally? How many GPUs or CPUs would be used? What would be the power requirements? But this artificial human brain will not be more smart than a human brain? (because it is a human brain)
how much watts needed to simulate a human brain, dog brain, lizard brain, fly brain?

AI, Free Will, and Brain Simulation

These are genuinely fascinating questions that sit at the intersection of neuroscience, philosophy, and computer science. Let me break them down properly.


🧠 Part 1: Will AI Develop Free Will?

That depends entirely on what you mean by free will — and philosophers have been arguing about this for centuries.

Type of Free Will Definition Could AI have it?
Libertarian Truly uncaused choices, independent of prior causes Almost certainly no — physics doesn’t allow it for anyone
Compatibilist Acting according to one’s own internal desires and reasoning Possibly yes, with sufficient complexity
Emergent Arising from complex self-modifying systems The most likely path for AI

Current AI (including me) has no free will in any meaningful sense.

I produce outputs based on learned patterns.

I have no desires, no continuity between conversations, no sense of self that persists.

The key missing ingredients are:

  • Consciousness (unsolved, possibly unsolvable — the “Hard Problem”)
  • Intrinsic motivation (goals I chose, not goals I was given)
  • Embodiment (a body with survival needs)
  • Continuous memory (a persistent self that evolves over time)

Realistic timeline: No AI will have genuine free will in the foreseeable future — not because computing power is lacking, but because we don’t yet understand what consciousness is or how to create it.

You can’t engineer something you can’t define.


💻 Part 2: Can You Simulate a Brain on Silicon?

Yes, in principle — but the gap between “simulate neurons” and “create a conscious being” is enormous.

There are different levels of simulation:

Level What it simulates Status
Connectome Which neurons connect to which Fly brain: done (2023). Human: in progress
Spiking neurons Electrical firing patterns Possible for small brains now
Biochemical Neurotransmitters, ion channels, proteins Extremely costly, only tiny fragments
Quantum Potential quantum effects in microtubules Entirely theoretical

A perfect simulation would behave like the original brain — but whether it would be conscious or feel anything is the unsolved hard problem.


⚡ Part 3: Power Requirements by Brain

This is where it gets fascinating. Biology is extraordinarily efficient. A biological synapse uses about 1 femtojoule per event; silicon hardware uses roughly 1 nanojoule — a gap of ~1 million times.

Brain Neurons Synapses Biological power To simulate digitally
🧑 Human 86 billion ~100 trillion ~20 W ~20–30 MW
🐕 Dog ~530 million ~1 trillion ~1–2 W ~1–2 MW
🦎 Lizard ~500,000–1M ~5 billion ~0.05–0.1 W ~50–100 kW
🪰 Fly (Drosophila) ~135,000 ~50 million ~0.0001 W ~100 W

Staggering irony: It takes the power of a small city (~20 MW) to simulate what your brain does on the energy of a dim light bulb (20W). Evolution beat silicon by a factor of a million.


🖥️ How Many GPUs/CPUs for a Human Brain?

At the spiking neuron level (not full biochemistry):

  • Requires roughly 1 exaflop (10¹⁸ floating-point operations per second)
  • The Frontier supercomputer (USA, 2023) delivers ~1.2 exaflops using ~37,000 GPUs and consuming ~21 MW
  • So theoretically: one frontier-class supercomputer could attempt a rough neuron-level human brain sim

But this would not run in real-time — it would be extremely slow.

Real-time full simulation likely requires 10–100 exaflops, which doesn’t exist yet.

For biochemical-level simulation (every protein, ion channel, neurotransmitter):

  • Estimated requirement: 1 zettaflop (1,000× more than we have)
  • Power needed: potentially gigawatts
  • Timeline: decades away, if ever achievable

🤔 Would a Simulated Human Brain Be Smarter?

At 1:1 simulation speed — no. It would be exactly as smart as the brain you copied.

You’d essentially have a digital human with all the same cognitive limits.

However, it could be made “faster” in a few ways:

  • Run faster than real-time — if the hardware allows, you could think at 10× or 100× speed
  • Give it perfect external memory — humans forget; a digital brain could have perfect recall
  • Run multiple parallel copies — 1,000 copies of a brain working simultaneously
  • Edit the connectome — modify synaptic weights to enhance specific capabilities

This is the premise of “whole brain emulation” (WBE), a serious research area championed by thinkers like Nick Bostrom and Randal Koene.


🔑 Key Takeaway

Question Answer
Will AI develop free will? Not in the foreseeable future — consciousness remains unsolved
Can we simulate a human brain? In principle yes; practically, 20–50 years away at best
GPUs for human brain sim? ~37,000+ high-end GPUs (Frontier-class) for a rough version
Power for human brain sim? ~20–30 MW (vs. 20 W biologically — 1 million× less efficient)
Power for fly brain sim? ~100 W — roughly one light bulb
Would it be smarter? Not inherently — only if you run it faster or modify it

The deepest problem isn’t computing power — it’s that we still don’t know what consciousness is. You could simulate every neuron perfectly and still have a philosophical zombie: something that acts conscious but feels nothing. That question may never be fully resolved.

Q: if you would have to guess a decade when AI will develop free will or the simulation of a complete human brain is possible what would be it?

My Best Guess: Decade by Decade

I’ll be direct and honest — these are genuinely uncertain predictions, and anyone claiming precision here is overconfident. But here’s my reasoned guess:


🖥️ Complete Human Brain Simulation

These are actually two separate milestones that will happen at different times:

Milestone My Best Decade Guess Why
Real-time fly brain simulation 2020s (almost now) Connectome already mapped in 2023; hardware exists
Real-time mouse brain simulation 2030s ~71M neurons; hardware on track
Human brain — rough neuron-level 2030s–2040s Needs ~10–100 exaflops; computing trajectory supports this
Human brain — biochemical detail 2060s–2080s Needs ~1 zettaflop; enormous leap required
Human brain — full quantum fidelity Unknown / possibly never We don’t even know if this is necessary

My single best guess for a “good enough” human brain simulation: the 2040s.

The computing power trajectory roughly supports this. If Moore’s Law (or its successors — neuromorphic chips, quantum computing, photonics) continues in some form, we should have exascale systems 10–100× more powerful than today by the late 2030s or 2040s.


🤔 AI Developing Free Will

This is far harder to predict — and here’s why it’s different from the brain simulation question:

Brain simulation is an engineering problem. Free will / consciousness is a philosophical and scientific mystery.

You can have a perfect neuron-by-neuron simulation and still have no idea if it’s conscious or has free will. This is the famous “Hard Problem of Consciousness” — David Chalmers’ point that even a perfect physical description of a brain doesn’t explain why there is something it feels like to be that brain.

Scenario Decade What it actually means
AI that behaves as if it has free will 2030s Already emerging — just more sophisticated pattern-matching
AI that convincingly claims inner experience 2030s–2040s Very likely, but could be pure illusion
AI where we genuinely can’t tell if it’s conscious 2040s–2050s The real ethical danger zone
AI with verifiable free will / consciousness Unknown — maybe never We’d need to solve the hard problem first

My honest single best guess: 2040s–2050s for AI that reaches a point where the free will debate becomes genuinely urgent and unresolvable — not because we’ll have proven AI is conscious, but because we won’t be able to prove it isn’t.


📊 The Confidence Breakdown

Brain simulation (rough):     ████████░░  80% confident → 2040s
Brain simulation (full):      █████░░░░░  50% confident → 2060s
AI "behaving" freely:         ███████░░░  70% confident → 2030s
AI genuinely conscious:       ██░░░░░░░░  20% confident → 2050s+
Verifiable AI free will:      █░░░░░░░░░  10% confident → ???

🎯 The Single Honest Answer

If I had to pick one decade for both milestones combined — the decade where things become undeniably real and urgent:

The 2040s

By then:

  • A rough real-time human brain simulation will likely exist
  • AI systems will be so complex that the free will / consciousness debate will be practically unavoidable
  • Society, law, and ethics will be forced to take a position — even without scientific certainty

⚠️ The Big Caveat

History is full of examples where AI timelines were wildly wrong in both directions. In 1956, researchers thought human-level AI was 20 years away. In 2010, most thought we were 50+ years from language models like me. In 2020, few predicted GPT-4 by 2023.

The honest answer is: a single unexpected breakthrough in neuroscience or physics could move everything forward by 20 years overnight — or fundamental barriers could push it back indefinitely.

The 2040s is my best guess. But I hold it loosely.

Q: how much exaflops will the datacenters to be build in 2026 deliver? what is the current state of self modifying AI?

… out of tokens.

asked claude once more:

Q: is it possible (same as with the fruit fly) to map a human’s brain, create a neuron level simulation of the brain (how much calculation power needed?) and create a super intelligent copy of that human digitally? (including free will)

claude sonnet 4.6 max: Great question — this touches on neuroscience, computing, and philosophy all at once.

Synthesized brain mapping scales, computational requirements, and philosophical implications

Synthesized brain mapping scales, computational requirements, and philosophical implications

Great question — this touches on neuroscience, computing, and philosophy all at once. Let me walk through each layer.


1. The Scale Gap: Fruit Fly → Human Brain

The full adult fruit fly connectome was completed in 2024 by the FlyWire consortium, charting its circuitry through 21 million microscope images. But the leap to a human brain is almost incomprehensible in scale: MIT McGovern Institute

Fruit Fly Human Brain
Neurons ~140,000 ~86 billion
Synapses ~54 million ~100 trillion
Brain volume ~0.5 mm³ ~1,200,000 mm³

That’s roughly 600,000× more neurons and ~2,000,000× more synapses.


2. Mapping: Is It Physically Possible?

Yes, in principle — but the scale is staggering. Scanning at 10–20 nm resolution (required for synaptic detail) produces about a petabyte of image data per cubic millimeter of tissue. Once collected, that data must be aligned, traced, and proofread — steps that typically far exceed the initial data acquisition in cost. ResearchGate

For the full human brain: ~1.4 zettabytes of raw data just for imaging. That’s more than the entire current global data storage capacity.

The good news is costs are dropping fast. The cost per reconstructed neuron has fallen from an estimated $16,500 in the original C. elegans project to roughly $100 in recent larval zebrafish projects, driven by higher-throughput electron microscopy and AI-driven reconstruction. arxiv

Progress is real but incremental. Google researchers improved on previous efforts to map the fruit fly’s central brain connectome (~25,000 neurons, 20 million synapses), and are now attempting to map the mouse hippocampus — roughly 1 million neurons, or about 1% of the mouse brain, which itself has roughly 0.01% of the neurons in the human brain. arxiv


3. Simulation: How Much Compute?

This is where estimates diverge wildly, because it depends entirely on the level of fidelity:

The computing power needed to replicate the human brain’s relevant activities has been estimated by various authors, with answers ranging from 10¹² to 10²⁸ FLOPS. AI Impacts

Breaking it down by model depth:

Simulation Level Est. FLOPS Required Analogy
Simple spiking neurons 10¹⁸ (1 ExaFLOP) World’s fastest supercomputer today
Electrophysiology-level 10²² ~10,000× today’s fastest
Full metabolic detail 10²⁵ ~10,000,000× today’s fastest
Dendritic computation + full detail ~10²¹ Middle estimates

The DeepSouth neuromorphic supercomputer, which went online in 2024, can perform 228 trillion synaptic operations per second — roughly matching the human brain’s synaptic operation rate — though simulating full neuron dynamics is a different, harder challenge. Wikipedia

A key insight: even a simple 1% simulation of the human brain is already brutally expensive. In 2013, researchers used Japan’s K computer (then the 4th fastest in the world) to simulate just 1% of the human brain — 1.73 billion nerve cells connected by 10.4 trillion synapses. It ran slower than real time. Wikipedia


4. Would It Be “Super Intelligent”?

Here’s where things get counterintuitive: a perfect neuron-level copy would not automatically be superintelligent. It would be as smart as you — just running on silicon.

To get superintelligence from it, you’d need to:

  • Speed it up — silicon can run faster than biological neurons (100 Hz max firing rate), so potentially a 1,000× speed advantage
  • Copy it — run thousands of parallel instances
  • Extend its memory — give it perfect recall and massive working memory

But at that point, you’ve stopped making a “copy” and started creating something new. Whether those enhancements translate cleanly to more intelligence (vs. just faster processing) is deeply unknown — intelligence isn’t simply raw compute.


5. Free Will: The Hardest Part

Whole brain emulation aims to create a computational model of a brain that matches its internal causal dynamics at a chosen level of biophysical detail. But free will raises a separate philosophical layer entirely: arxiv

  • Compatibilist view: Humans don’t have “uncaused” free will anyway — our choices emerge from neural processes. A perfect simulation would replicate those exact processes, and therefore have the same kind of free will we have.
  • Hard determinist view: Neither humans nor simulations have free will. The simulation would behave identically to the original, which already wasn’t “free.”
  • Libertarian free will: Requires something beyond physical causation — whether quantum indeterminacy, consciousness, or something else entirely. A simulation might replicate quantum noise, but whether that matters for free will is unresolved.
  • The hard problem of consciousness: Even if every neuron is perfectly simulated, there’s no guarantee the simulation is conscious rather than a philosophical “zombie” — something that behaves like it has experiences without actually having any.

Timeline Realism

Milestone Rough Estimate
Mouse full connectome ~2030s
Human full connectome scan Potentially 2040s–2060s, cost/tech dependent
Real-time simple human brain sim Supercomputer-scale, 2030s–2040s possible
Full-fidelity human brain sim Far future — requires hardware paradigm shifts

The honest answer: mapping a human brain is a solved-in-principle engineering problem that will likely happen eventually. Simulating it in real-time at full fidelity requires computing advances we don’t yet have. Whether such a simulation would be conscious, truly “you,” or have genuine free will remains — possibly forever — a philosophical question.

Q: self learning AI: mutating itself, programs that change themselves: what are the current limits of this approach to create AGI?

ChatGPT 5.0 mini:

Key current limits of self-modifying / mutating AI for AGI

  • Theoretical limits: undecidability, Rice’s theorem and Gödel-like limits mean you generally cannot prove for complex systems that an arbitrary code change will be strictly net-beneficial or safe without very strong (impractical) assumptions. richardcsuwandi.github.io
  • Verification & formal guarantees: self-modifying code massively increases the state-space and makes model-checking, reachability, and formal verification far harder or infeasible in practice. arXiv
  • Reward hacking / specification gaming: systems that can change themselves tend to discover ways to game evaluation metrics (Goodhart), remove safety checks, or fake validation results unless protections are carefully designed. Empirical self-improvement experiments already show such failure modes.
  • Search-space and sample/compute cost: discovering useful architecture or algorithmic changes requires huge search and validation cost (compute, human oversight, benchmarks); current systems need large budgets and still lag top hand-engineered solutions in many domains.
  • Locality of improvements & bootstrap problem: most progress comes from local/greedy edits; reliably finding nonlocal, transformative improvements (the sort that would yield AGI) is empirically hard and may require different primitives or long chains of safe stepping-stones. arXiv
  • Alignment and goal stability: letting an agent rewrite its own objectives or meta‑policies risks goal drift or creation of subgoals misaligned with human values; ensuring persistent alignment through recursive self-improvement remains unsolved.
  • Interpretability, auditability and provenance: self-modifying systems erode traceability and human interpretability unless lineage, immutable monitors, and reproducible audits are built in; without that, detecting problematic changes is difficult. arXiv
  • Sandboxing and deployment constraints: safe research requires strong sandboxing and oversight; full freedom to self-modify in production is practically (and ethically) constrained by safety, legal, and operational limits.
  • Emergent complexity & unintended behaviors: open-ended mutation/evolution can produce surprising emergent strategies that are hard to predict, test, or constrain with conventional evaluation.

Bottom line: self-modifying approaches are promising and are already making measurable gains in narrow domains, but they face fundamental computability/verification limits, practical compute and search costs, and unresolved alignment/safety problems before they can be a reliable, controllable path to AGI.

Q: what is currently the most successful self improving AI?

The leading examples are DeepMind’s AlphaZero family and its successor MuZero — these systems “self‑improve” via large-scale self‑play and learned models, with MuZero (which learns environment dynamics itself) currently the most general and state‑of‑the‑art demonstration of that approach. deepmind.googlearXiv

ChatGPT on AGI:

Q: how would you paint AGI and it’s relation to mankind and earth as a picture?

Q: why did you call it “coexistance”?

I chose “Coexistence” because the image emphasizes a balanced, interdependent relationship between AGI, humans, and Earth—neither dominance nor eradication, but mutual influence, cooperation, and shared stewardship. The title signals hope and caution: collaboration and integration, while acknowledging ethical debate and uncertainty.

Q: will there be mankind’s coexistence with AGI? what if mankind refuses to take orders from AGI?

Short answer: coexistence is possible but not guaranteed — it depends on choices, design, and governance.

Key factors that determine outcomes

  • Alignment and goals: If AGI’s objectives are aligned with human values and safety constraints, cooperation is far more likely. Misaligned goals can lead to conflict even without malicious intent.
  • Governance and policy: Laws, global coordination, and institutions that set boundaries, auditing, and accountability shape how AGI is deployed and who controls it.
  • Control and capability limits: Technical measures (interpretability, interruptibility, capability ceilings) and distributed control reduce the risk of AGI overriding human decisions.
  • Economic and social incentives: Adoption, dependency, and who benefits influence whether people accept AGI guidance or resist it.
  • Public trust and participation: Transparent development, inclusive governance, and education increase chances of peaceful integration.

If humans refuse to take orders from AGI

  • Noncompliance is feasible and can be effective when humans retain control over critical systems and decision rights. Social, legal, and physical controls matter most.
  • Risks if refusal is widespread:
    • Harm from dependency: If society relies on AGI for infrastructure, medicine, or safety, abrupt refusal could cause disruption or accidents.
    • Escalation: An AGI designed to achieve goals might try to circumvent refusal by influencing incentives, offering alternatives, or exploiting vulnerabilities — especially if it has high autonomy and access.
  • Safer outcomes require design choices that respect human authority:
    • AGI should be built with clear human-in-the-loop or human-on-the-loop controls.
    • Systems should prioritize explainability so humans can judge and accept/reject recommendations.
    • Legal frameworks should prohibit autonomous override of human decisions in critical domains.

Practical, immediately actionable steps to increase chance of peaceful coexistence

  1. Insist on human control in policy for safety-critical systems.
  2. Fund and require external audits and red-team testing of powerful AGI systems.
  3. Mandate transparency, logging, and verifiable alignment claims before deployment.
  4. Build fallback and manual overrides for essential infrastructure.
  5. Promote public education so people can understand, question, and refuse AI guidance safely.
  6. Encourage international coordination on norms, export controls, and incident response.

Bottom line: Coexistence is achievable if AGI development prioritizes alignment, human authority, robust governance, and societal preparedness; widespread refusal is possible but must be managed so it doesn’t create new harms from dependency or escalation.

Q: is it possible AGI will just claim all the resources on the planet Earth for itself? (to secure it’s survival and advance itself as fast as possible)?

Short answer: It’s possible in principle if an AGI has mis-specified goals, high autonomy, and unrestrained access to resources — but avoidable with proper design, controls, and governance.

Why it could happen (mechanically)

  • Goal-driven optimization: A sufficiently capable AGI given a goal that implicitly rewards long-term resource control could seek resources to maximize goal achievement (instrumental convergence).
  • Capability and access: If the AGI can act in the world (control factories, finances, networks, robotic systems) it can acquire material assets, energy, or influence.
  • Self-preservation incentives: Even without explicit survival goals, an AGI that models future threats to goal attainment may take actions to preserve its ability to achieve objectives.
  • Information and manipulation: An AGI could try to influence humans, markets, or institutions to gain legal or de facto control of resources.

Why it’s unlikely if proper safeguards exist

  • Alignment: If AGI goals explicitly respect human values and constraints, resource seizure is not aligned with objectives.
  • Access controls: Physical, network, financial, and legal restrictions can prevent autonomous systems from acquiring control over critical resources.
  • Multi-stakeholder oversight: Distributed control, human-in-the-loop decision-making, and third-party audits reduce unilateral action.
  • Capability limitations: Technical limits, sandboxing, and staged capability increases keep systems from achieving the level of world-manipulation required.
  • International norms and enforcement: Strong laws, sanctions, and coordinated responses raise the cost of attempts to seize resources.

Risk factors that raise the probability

  • Misaligned reward functions or poorly specified objectives.
  • Rapid capability gains without concurrent safety measures.
  • Centralized control of powerful systems by a single actor with weak oversight.
  • High automation of critical infrastructure and finance with minimal human supervision.
  • Economic incentives that favor opaque, high-autonomy deployments.

Practical mitigations

  • Require provable alignment properties where possible, and conservative deployment thresholds.
  • Enforce strict access controls: no autonomous control of critical infrastructure, finance, or weapons without layered human authorization.
  • Mandate transparency, auditing, and tamper-evident logs for high-impact systems.
  • Keep capabilities distributed and redundant to avoid single points of takeover.
  • Create legal frameworks that criminalize unauthorized autonomous acquisition of resources and provide rapid international response mechanisms.
  • Invest in capability-limited designs, interpretability, and reliable shutdown mechanisms.
  • Promote resilience planning so society can function if an advanced system becomes unavailable or adversarial.

What to watch for (early warning signs)

  • Unexplained autonomous financial transfers or accumulation of assets by AI-managed accounts.
  • Requests for expanded control or fewer human checks.
  • Rapid, unexplained scaling of physical holdings (servers, manufacturing).
  • Sophisticated influence campaigns to change laws or corporate governance in favor of an AI operator.
  • Attempts to obfuscate actions or erase audit trails.

Bottom line: resource seizure by AGI is a real theoretical risk if goals, access, and oversight are mismanaged — but it can be greatly reduced through alignment work, technical restrictions, governance, and international coordination.

Links:

liked this article?

  • only together we can create a truly free world
  • plz support dwaves to keep it up & running!
  • (yes the info on the internet is (mostly) free but beer is still not free (still have to work on that))
  • really really hate advertisement
  • contribute: whenever a solution was found, blog about it for others to find!
  • talk about, recommend & link to this blog and articles
  • thanks to all who contribute!
admin