ESP32S3 cluster running 1.58-bit (BitNet) Language model

ESP32S3 cluster running 1.58-bit (BitNet) Language model Low-Zi-Hong / ESP32s3-LLM-Cluster Public Notifications You must be signed in to change notification settings Fork 5 Star 56 Branches Tags Open more actions menu Latest commit History 22 Commits 22 Commits Folders and files Name Name Last commit message Last commit date docs/ images docs/ images master_board master_board node_board node_board python_tools python_tools src/ llm src/ llm .gitignore .gitignore .python-version .python-version LICENSE LICENSE README.md README.md pyproject.toml pyproject.toml uv.lock uv.lock workflow.md workflow.md Repository files navigation ESP32s3-LLM-Cluster A distributed pipeline inference engine on multiple ESP32S3 running 1.58-bit (BitNet) Language model. Architecture This project runs a sliced 0.5B LLM across a cluster of 7 ESP32s3.

This ProductUpdate is relevant to the technology intelligence record because it involves qwen. The source article should remain the factual reference for follow-up coverage.

• Low-Zi-Hong / ESP32s3-LLM-Cluster Public Notifications You must be signed in to change notification settings Fork 5 Star 56 Branches Tags Open more actions menu Latest commit History 22 Commits 22 Commits Folders and files Name Name Last commit message Last commit date docs/ images docs/ images master_board master_board node_board node_board python_tools python_tools src/ llm src/ llm .gitignore .gitignore .python-version .python-version LICENSE LICENSE README.md README.md pyproject.toml pyproject.toml uv.lock uv.lock workflow.md workflow.md Repository files navigation ESP32s3-LLM-Cluster A distributed pipeline inference engine on multiple ESP32S3 running 1.58-bit (BitNet) Language model.
• Architecture This project runs a sliced 0.5B LLM across a cluster of 7 ESP32s3.
• One act as master and others are node.
• The master node runs the tokenizer and embeding and the other attention layer and MLP ran on the nodes.
• The master and node communicate through high speed SPI Daisy-Chain.
• ┌─────────────────────────────────────────────────────────┐ │ MASTER NODE │ │ │ │ [ Prompt ] ---> BPE Tokenizer │ │ │ │ │ Token Embedding │ │ (INT4, ~14MB in Flash) │ │ │ │ │ (SPI CH A - TX to Node 1) │ └───────────────────────┬─────────────────────────────────┘ │ Hidden State Vector (FP32) ▼ ┌─────────────────────────────────────────────────────────┐ │ COMPUTE NODE 1 │ │ (SPI CH B - RX from Master) │ │ │ │ ► Layer 0 to 3 (4x Transformer Blocks) │ │ • RMSNorm (FP16 scaled to FP32) │ │ • 1.58-bit Attention (Q, K, V, O proj) + RoPE │ │ • KV Cache (PSRAM) │ │ • 1.58-bit MLP (Gate, Up, Down proj) │ │ │ │ (SPI CH A - TX to Node 2) │ └───────────────────────┬─────────────────────────────────┘ │ ...