# WebGPU + OpenRouter BYOK Project Ideas

> **Baseline Reference**: `shader.html` — AI-powered WebGPU WGSL shader playground with live preview, static HTML, client-side OpenRouter API key (BYOK), no backend required.

---

## Table of Contents

- [Core Architecture Patterns](#core-architecture-patterns)
- [Visual Tools](#visual-tools)
- [Games](#games)
- [Apps](#apps)
- [Interactive Visualizations](#interactive-visualizations)
- [Game / Simulation Hybrids](#game--simulation-hybrids)
- [AI + GPU Hybrid Tools](#ai--gpu-hybrid-tools)
- [Data / Analytical Tools](#data--analytical-tools)
- [Generative Art & Creative Coding](#generative-art--creative-coding)
- [Experimental / Research Ideas](#experimental--research-ideas)
- [Meta Tools (for developers)](#meta-tools-for-developers)
- [Reusable Primitives](#reusable-primitives)
- [Implementation Notes](#implementation-notes)

---

## Core Architecture Patterns

All projects share these reusable primitives:

### 1. Prompt → WGSL
- **Core engine**: OpenRouter LLM (Claude, GPT-4o, Gemini, Qwen, Llama)
- Client-side API key (BYOK), stored in `localStorage`
- System prompt enforces WGSL-only output (no markdown, no explanations)

### 2. WGSL → WebGPU Pipeline
- Vertex + fragment shaders for rendering
- Compute shaders for simulations/data processing
- Pipeline configuration (blend modes, render targets, uniforms)

### 3. Output → Visualization
- Canvas rendering (2D/3D)
- Babylon.js / Three.js WebGPU renderer for 3D scenes
- Data visualization (charts, heatmaps, graphs)
- UI components rendered via shaders

> **Everything else is just composition of these three primitives.**

---

## Visual Tools

### 1. AI Compute Shader Playground
Like shader.html but for WebGPU **compute shaders** (physics simulations, data processing, fluid dynamics).
- Prompt → OpenRouter generates WGSL compute code
- Live preview of simulation results (particle systems, fluid dynamics)
- Same stack: static HTML, client-side OpenRouter, WebGPU compute pipeline

### 2. Procedural Texture Generator
Prompt → LLM generates WGSL fragment shaders for seamless textures (wood, marble, noise patterns).
- Render on 3D primitives (cube, sphere) with WebGPU
- Export textures as PNG/KTX2
- Add parameter sliders (speed, scale) to tweak generated shaders

### 3. Node-Based Shader Editor (AI-Assisted)
Visual node graph to wire shader components, with "AI Generate Node" button.
- OpenRouter generates WGSL code for custom nodes from text descriptions
- Export final shader as WGSL/GLSL
- No backend needed

### 4. Post-Processing Effect Studio
Prompt → LLM generates WGSL post-processing shaders (bloom, color grading, blur).
- Apply to a base 3D scene (Babylon.js/Three.js WebGPU renderer) with live toggle
- Save/load effect presets to localStorage

### 5. AI Material Generator (Physically-Based Rendering) 🆕
Prompt → "rusty metal with scratches and oil stains"
LLM generates:
- WGSL shader
- PBR parameters (roughness, metallic, normal maps logic)
- Apply to 3D models in a viewer (like a mini Blender material panel)

> 👉 This becomes a **Substance Designer-lite in the browser**

### 6. Style Transfer Shader Lab (Real-Time) 🆕
Upload image → LLM generates shader approximating a style (impressionism, cyberpunk).
- WebGPU applies transformation in real time
- Compare original vs processed

> 👉 No heavy ML model—just procedural approximation via shaders

### 7. AI Lighting Designer 🆕
Prompt → "sunset cinematic lighting with soft shadows"
LLM generates:
- Light setup
- Shadow shader tweaks
- Tone mapping curve

> 👉 Think lighting tool like in Unreal Engine but AI-driven

### 8. GPU UI Renderer (Experimental) 🆕
Entire UI rendered via shaders (buttons, panels, animations).
- Prompt → "glassmorphism UI with neon glow"
- LLM generates fragment shaders for UI components

> 👉 This is very bleeding edge: replacing DOM/CSS with GPU rendering

---

## Games

### 9. AI-Generated 2D Game Prototype
Prompt → OpenRouter generates game logic (player movement, enemy behavior, win conditions).
- WebGPU renders 2D sprites/scenes with WGSL shaders for visual effects
- Minimal example: Pong with LLM-generated paddle/ball behavior shaders

### 10. Procedural Level Generator
Prompt → LLM generates level layout descriptions (platform positions, enemy spawns).
- WebGPU renders the level with generated shaders for each level element
- Save generated levels to localStorage, no server needed

### 11. Shader-Based Game (Breakout Clone)
All visual elements (ball, paddle, blocks) use WGSL shaders generated via OpenRouter.
- Prompt per element: "glowing neon ball shader", "cracked brick shader"
- Static HTML, WebGPU rendering, no backend

---

## Apps

### 12. AI-Powered 3D Model Viewer
Prompt → OpenRouter generates glTF/OBJ model descriptions or simple geometry code.
- Render with WebGPU (use Babylon.js/Three.js WebGPU renderer) with generated material shaders
- Drag-and-drop model support, all client-side

### 13. Data Visualization Generator
Prompt → LLM generates WebGPU compute shaders to process CSV/JSON data.
- Render interactive charts/graphs (bar, line, scatter) with custom WGSL shaders for styling
- Export visualizations as PNG/SVG

### 14. WebGPU Benchmark Tool (AI-Powered)
OpenRouter generates stress test shaders (heavy fragment/compute workloads).
- Measure FPS, GPU memory usage, WebGPU feature support
- Save benchmark results to localStorage, compare across devices

---

## Interactive Visualizations

### 15. AI-Generated Particle System Simulator
Prompt → LLM generates WGSL compute shaders for particle behavior (gravity, collision, color).
- WebGPU renders millions of particles at 60fps, live parameter tweaking
- Save/share particle presets via URL hash (no backend)

### 16. Real-Time Ray Tracing Demo (AI-Enhanced)
Prompt → LLM generates WGSL ray tracing shaders (spheres, reflections, soft shadows).
- Use WebRTX (WebGPU Ray Tracing Extension) for real-time RT
- Toggle AI-generated scene descriptions (objects, lighting) via OpenRouter

### 17. Fluid Simulation Playground
Prompt → LLM generates WGSL compute shaders for Navier-Stokes fluid dynamics.
- WebGPU computes and renders fluid in real-time, pointer interaction to add forces
- Adjust simulation parameters (viscosity, speed) via UI sliders

---

## Game / Simulation Hybrids 🎮

### 18. AI Physics Sandbox (Compute Shader Driven) 🆕
Prompt → "sand simulation with wind and erosion"
LLM generates compute shaders for:
- Particle rules
- Grid updates

> 👉 Like a programmable version of falling sand games

### 19. Ecosystem Simulator 🆕
Prompt → "predator-prey ecosystem with migration"
Compute shaders simulate:
- Agents
- Reproduction
- Movement

> 👉 Visualization + simulation entirely GPU-side

### 20. AI Terrain Generator + Renderer 🆕
Prompt → "alien desert with floating islands"
LLM generates:
- Heightmap logic (noise functions)
- Terrain shader

> 👉 Combine procedural gen + rendering in one pipeline

---

## AI + GPU Hybrid Tools 🧠

### 21. Prompt-to-Visualization Compiler 🆕
Input: "show how a neural network learns XOR"
LLM outputs:
- Compute shader for training steps
- Rendering shader for visualization

> 👉 Educational + technical hybrid

### 22. Code-to-Shader Visualizer 🆕
Paste JS/Python code.
LLM converts logic → visual shader representation.

Examples:
- Sorting algorithm → animated bars
- Graph traversal → animated nodes

### 23. AI Debug View Generator 🆕
Prompt → "highlight overdraw" or "visualize normals"
LLM injects debug shaders into pipeline.

> 👉 Like dev tools but AI-driven

---

## Data / Analytical Tools 📊

### 24. Massive Dataset Explorer (GPU-first) 🆕
Load CSV (millions of rows).
LLM generates:
- Compute shader aggregations
- Rendering shader

> 👉 Beyond D3.js scale

### 25. AI Heatmap Generator 🆕
Prompt → "density map with smooth falloff and color ramp"
LLM generates fragment shader for visualization

---

## Generative Art & Creative Coding 🎨

### 26. Prompt-to-Generative Art Engine 🆕
Prompt → "minimalist black and white geometric animation"
LLM generates animated shader.

> 👉 Feels like evolving Shadertoy into AI-first

### 27. Audio-Reactive Shader Generator 🆕
Input: microphone or music file.
Prompt → "bass-driven pulsing neon waves"
LLM generates shader using FFT data

### 28. AI VJ Tool (Live Visuals) 🆕
Prompt → live generation of visuals during performance.
Switch shaders on the fly.

> 👉 Could replace parts of tools like Resolume

---

## Experimental / Research Ideas 🧪

### 29. Evolutionary Shader Lab 🆕
- Generate multiple shaders via LLM
- User selects favorites
- System mutates them

> 👉 "genetic algorithm for visuals"

### 30. Constraint-Based Shader Generator 🆕
Input:
- "must be under 50 lines"
- "must run at 60fps"
LLM generates optimized shader

### 31. Cross-API Shader Translator 🆕
Input: GLSL → Output: WGSL / HLSL / Metal

> 👉 Useful bridge between WebGL and WebGPU

---

## Meta Tools (for developers) ⚙️

### 32. WGSL Linter + Fixer (AI-powered) 🆕
Detect:
- Invalid bindings
- Performance issues
- Auto-fix via LLM

### 33. Shader Performance Optimizer 🆕
Input shader → LLM rewrites for:
- Fewer ops
- Better memory usage

### 34. WebGPU Pipeline Generator 🆕
Prompt → "render particles with additive blending"
LLM outputs:
- Full JS + WGSL pipeline code

---

## Implementation Notes

### Tech Stack (Proven by shader.html)
- **Static HTML**: No backend required, host on GitHub Pages/Netlify for free
- **OpenRouter BYOK**: Client-side API calls work (as proven by shader.html), store API key in `localStorage`
- **WebGPU**: Supported in Chrome 113+, Edge, Safari 26+ (with flag), no plugins needed
- **No CORS issues**: OpenRouter API allows client-side requests (shader.html already works)

### Key Implementation Patterns (from shader.html)
```javascript
// 1. API Key Handling
localStorage.setItem('or_api_key', apiKey);
const savedKey = localStorage.getItem('or_api_key');

// 2. OpenRouter Request
const response = await fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Authorization": `Bearer ${apiKey}`,
    "HTTP-Referer": "https://yoursite.com",
    "X-Title": "Your App Name",
  },
  body: JSON.stringify({
    model: model,
    messages: [{ role: "system", content: "..." }, { role: "user", content: prompt }],
    temperature: 0.7,
    max_tokens: 1024
  })
});

// 3. WebGPU Init
const adapter = await navigator.gpu.requestAdapter();
const device = await adapter.requestDevice();
const context = canvas.getContext("webgpu");
context.configure({ device, format: navigator.gpu.getPreferredCanvasFormat() });

// 4. Shader Compilation & Rendering
const shaderModule = device.createShaderModule({ code: VERTEX_SHADER + fragmentCode });
const pipeline = device.createRenderPipeline({ /* ... */ });
// ... render pass setup and submit
```

### System Prompt Template for LLM
```
You are a WGSL (WebGPU Shading Language) expert.

RULES:
- Output ONLY raw WGSL code. No markdown, no explanations, no comments about the code.
- You MUST define: @fragment fn fs_main(@builtin(position) pos : vec4<f32>) -> @location(0) vec4<f32>
- Use 'pos' parameter for UV/pixel coordinates
- Return vec4<f32>(r, g, b, 1.0) as final color
- Keep it simple and visually interesting
- [Project-specific rules here]
```

---

## Resources

- **shader.html**: Baseline reference implementation at `/mnt/apps/BACKUP-RAG/clickandbuilds/Applications/shader.html`
- **WebGPU Specification**: https://www.w3.org/TR/webgpu/
- **WGSL Spec**: https://www.w3.org/TR/WGSL/
- **OpenRouter API Docs**: https://openrouter.ai/docs/quickstart
- **Awesome WebGPU**: https://github.com/mikbry/awesome-webgpu
- **WebGPU Samples**: https://github.com/webgpu/webgpu-samples
- **Babylon.js WebGPU**: https://doc.babylonjs.com/setup/support/webGPU
- **Three.js WebGPU Renderer**: https://threejs.org/docs/#manual/en/introduction/WebGPURenderer

---

*Document generated on 2026-05-05 | Covers 34 project ideas + shader.html baseline*