Neural Computers (The Data Pipeline)

📘 Blog   ·   🎬 Demo Gallery   ·   📄 Paper

News

• 2026 April: First open-source release of Neural Computers, starting with the data pipeline for CLI and GUI trajectory generation.

Abstract

Neural Computers (NCs) are neural networks that unify computation, memory, and I/O in a single latent runtime state. The long-term goal is the Completely Neural Computer (CNC): a general-purpose neural computer that challenges the layered architecture of conventional computers. As an initial step, this project studies whether executable dynamics can be learned solely from collected I/O traces, without access to instrumented program state, using CLI and GUI trajectories as training and evaluation data.

Quick Start

Environment Python 3.9+ · Docker · asciinema · agg · ffmpeg

1. CLI / Asciinema

python3 main.py cligen asciinema record --command "python3 --version"
python3 main.py cligen asciinema cast-to-mp4 workspace/cligen_general/casts

2. CLI / VHS

python3 main.py cligen vhs build-image
python3 main.py cligen vhs generate-basic --count 10 --prefix demo
python3 main.py cligen vhs make-manifest
python3 main.py cligen vhs run-manifest

3. GUI / Synthetic

pip install -r engine/gui/synthetic_data_collection/requirements.txt
python3 main.py guiworld build-image
python3 main.py guiworld synthetic --count 1 --max-workers 1

Runtime Generations from Neural Computers

_{Using training data generated by the code released in this repository.}

CLIGen (general / asciinema)

CLIGen (clean / vhs)

GUIWorld (GUI)

_{Alternating: Conventional Computer / Neural Computer}

Acknowledge

Our models (not released here) are built from Wan2.1, Matrix-Game-2. The data engine for CLIGen are built from Asciinema, VHS, and data engine for GUIWorld are modified directly from Neural-OS.