Ruido_Visual_Web-VJ_Project

Ruido Visual Lab Web-VJ (RVWVP)

Español

WebGL2-based VJ application, inspired by Resolume 2.4, that lets you mix and manipulate video in real time with visual effects and projection mapping. Built as a self-contained static website: no frameworks, no build tools, no Node.js dependencies. Just vanilla JavaScript with ES Modules.

License: GPL-3.0 · Version: 0.3.11

📘 To start using it, see the User Manual.

---

Table of contents

• Overview and features
• Running
  • Local
  • Web (production)
  • VPS / server
• Architecture
• Mapping Mode
• Project structure
• Additional documentation
• Released versions
• License

---

Overview and features

RVWVP is a VJ software that runs entirely in the browser. It mixes dynamic video layers in real time (with per-layer blend modes and RGB tint), applies the catalog's visual effects (40; the count comes from js/effect-registry.js) per layer or globally, and projects the output to an external window/monitor. It also includes a Mapping Mode that lets you project video onto polygonal regions of a facade image.

Features

• Dynamic video layers (0.3.2) with independent individual effects (video file or webcam): starts with 3 layers, add with '+ Capa' — the new layer comes in on top, so you build forwards (layer 1 as background and the next ones above it) instead of moving it up by hand —, delete with ✕; advised to stay at or below 5 (warning, not a block)
• Blend modes (Normal, Add, Multiply, Screen, Overlay, Difference) and per-layer RGB tint (0.3.1)
• Global layer (formerly Global FX) — post-composition effects over the full output, with its own blend mode, opacity and tint to combine its result with the composition
• 40 visual effects grouped as in the interface (the full list lives in js/effect-registry.js):
  • Per-layer/global effects: Chroma Key, Mirror, Kaleidoscope, Feedback, Rotation, Blur, Color Shift, Invert, Zoom, Flip, Duotone, Luma Key, Posterize, Dither, Bloom, Contour, Radial blur, Mask, Domain Warp
  • Visual textures: Dots, Grid, Scanlines, Vignette, RGB Shift, Shake, Pixelate, TYPO (text with fonts), ASCII, Wave, Tunnel, Glitch
  • Generative textures: Plasma, Voronoi, Noise, Fractal (Julia set), Metaballs, Starfield, Oscilloscope — the seven that create the image on their own: the pattern is the layer's image, and how it combines (cover, add, blend) is decided by its blend mode and opacity. The rest of the catalog needs an image to start from
  • Cinematic movements: Motion Control (2D layer transform with keyframe recording/playback) and Frame (editable layer rectangle)
• TYPO / ASCII module — visual text and ASCII art with loadable fonts from the local typo/ folder (FontFace API; monospace fallback)
• Mapping Mode — projection mapping onto facades with up to 3 polygonal regions and layer assignment
• MIDI control — dynamic parameter mapping per layer or global layer, independent of the SEL button; includes a standard MIDI mapping in docs/
• Fader automation — automatic value cycling (loop/bounce mode)
• Projector window — independent output to an external screen/projector via window.postMessage
• Audio system — audio track playback with real-time spectral analysis:
  • Band analysis: RMS, bass, mids, treble, and peak
  • Lightweight BPM detection via bass peaks + autocorrelation, with tap tempo and manual override
  • WebAudio effects: low-pass filter, delay, reverb, and gain
  • Advanced audio popup window with spectrum/waveform visualization
  • (It is a player/analyzer — basic sequencer —, not a DAW: the VJ creates the effects to generate the sense of video-audio sync)
• Audio→video modulation — mapping of audio parameters (RMS, bass, BPM...) to any visual parameter, with add / temporal override / replace modes
• Video recording — canvas capture to a WebM/MP4 file
• Video+audio recording — synchronized recording of the canvas and the processed audio output (recordings can be reused as samplers of your remixes)
• Project save/load — configuration persistence (effects, MIDI, mapping, audio, modulation)
• Performance monitor — real-time FPS metrics

Tech stack

Technology	Use
WebGL2	Graphics and shader rendering on GPU
Web MIDI API	Support for physical MIDI controllers
Web Audio API	Audio playback, spectral analysis, effects, and BPM detection
Canvas Recording API	Video export/recording
getUserMedia	Webcam input as a video layer
FontFace API	Loading TYPO/ASCII fonts from typo/
Vanilla JS (ES Modules)	Application logic — no frameworks

Supported browsers: Chrome 90+, Firefox 88+, Safari 15+, Edge 90+ (require WebGL2). Recommended: Firefox (the testing browser). If something fails in your browser, report it to contacto(at)ruidovisualab.com.

---

Running

Important: Do not open index.html directly from the file system. ES Modules require an HTTP server due to browser CORS restrictions.

Local

Option A — Start script (recommended):

Linux/Mac:

./start-server.sh

Windows:

start-server.bat

The script detects Python, starts a server on port 8000, and tries to open the browser automatically.

Option B — Manual server:

python3 -m http.server 8000

Then visit: http://localhost:8000

Web (production)

You can use the production version published at https://ruidovisualab.com without installing anything.

Production (VPS / server)

Since this is a static website, deployment simply consists of copying all files (index.html, mapping.html, projector.html, audio.html, favicon.ico, js/, typo/, LICENSE) to the public directory of any web server. There is no build step.

With Nginx:

1. Copy the files to the server:

   scp -r ./* user@your-vps:/var/www/rvwvp/
   

2. Configure a server block in Nginx:

   server {
   listen 80;
   server_name rvwvp.yourdomain.com;   # or the VPS IP
   root /var/www/rvwvp;
   index index.html;
   
   # Correct MIME types for ES modules
   location / {
       try_files $uri $uri/ =404;
   }
   
   # Aggressive caching for JS modules (immutable between releases)
   location /js/ {
       add_header Cache-Control "public, max-age=31536000";
   }
   
   # Do not cache HTML (so changes show up instantly)
   location ~ \.html$ {
       add_header Cache-Control "no-cache";
   }
   }
   

3. Reload Nginx:

   sudo nginx -t && sudo nginx -s reload
   

With Apache: copy the files to the DocumentRoot (e.g. /var/www/html/rvwvp/). Apache serves .js files with the correct MIME type by default.

With any static server (Caddy, serve, http-server, Netlify, Vercel...): just serve the project root directory.

Note on pop-up windows: the projector and the mapping editor use window.open(). Some browsers block pop-ups unless they originate from a user gesture. The "Abrir Proyector" and "Mapping" buttons are direct user clicks, so they work; if your browser blocks them, add the domain to the pop-up exceptions.

---

Architecture

Core pattern: Effect Registry

The core of the design is the Effect Registry Pattern: a single file (js/effect-registry.js) is the single source of truth for the metadata of all effects —parameters, ranges, value transforms, shader uniform mappings—. Derived from this registry automatically: MIDI-mappable parameters, fader automation configs and the initial per-layer effect state. The slider UI is NOT generated automatically: each card lives by hand in index.html (a data-effect button and data-effect-param controls that match the registry's sliderId).

Adding a new effect requires touching: its registry entry, its fragment in shader-library.js, the compile line in compileBasicShaders() (webgl-renderer.js), its id in EFFECT_ORDER with an apply<Id> method (or isGenerative) in composition-service.js, and its card in index.html. The step-by-step recipe lives in the "Añadir un efecto nuevo" route of the code tour.

Rendering pipeline (normal mode)

Video frames (N layers)
    │  getLayerTextures(gl) — raw texture per layer (UNPACK_FLIP_Y)
    ▼
renderer.processLayers()
    │  applies per-layer effects via dedicated ping/pong FBOs
    ▼
renderer.composeLayers()
    │  back-to-front shader-based composition (blend modes, opacity, RGB tint)
    ▼
Global layer (optional)
    │  post-composition effects; its output composes by its mode, opacity and tint
    ▼
Global intensity (optional)
    │  RGB multiplier 0.0–2.0
    ▼
renderToScreen()  →  #output-canvas (1280×720)
    │
    └─→ transferable createImageBitmap() ~60fps → projector.html
    └─→ captureStream() → MediaRecorder → WebM/MP4 recording

Mapping Mode branch

When mappingMode is active, render() switches routes: instead of normal composition, it calls mappingRenderer.renderToCanvas(), which draws the facade image and projects each layer's video inside its assigned polygon (clipping by triangulation, affine mapping). Per-layer effects are still processed (processLayers with ensureFBOTextures), so the mapping respects the effects applied to each layer.

For a detailed explanation of each module and the data flow, see documentation.md.

---

Mapping Mode

Mapping Mode projects video onto polygonal regions of a facade image: you load the photo, draw up to three polygons, assign each one to a layer, and the output shows the facade with the video clipped and mapped inside its polygon. The editor opens with the "Mapping" button in the top bar and the mode keeps projecting when you close its window.

The step-by-step walkthrough is in the user manual §4; the protocol between the editor window and the main one is in documentation.md.

---

Project structure

rvwvp/
├── index.html              # Main control interface
├── audio.html              # Advanced audio popup window (spectrum, mappings)
├── mapping.html            # Mapping editor window (full screen)
├── projector.html          # External projection window
├── favicon.ico             # Site icon (at the root: that is where the browser asks)
├── start-server.sh         # Start script (Linux/Mac)
├── start-server.bat        # Start script (Windows)
├── LICENSE                 # GPL-3.0 (official English text)
├── LICENSE.es              # License summary in Spanish
├── README.md               # This README (Spanish)
├── README.en.md            # This README (English)
├── manual_de_usuario.md    # User manual (Spanish)
├── user_manual.md          # User manual (English)
├── documentacion.md        # Detailed technical documentation (Spanish)
├── documentation.md        # Detailed technical documentation (English)
├── docs/
│   ├── preparacion_de_videos.md   # Video preparation guide (Spanish)
│   ├── video_preparation.md       # Video preparation guide (English)
│   ├── historial_de_refactorizacion.md # Refactoring history (Spanish)
│   ├── refactor_history.md        # Refactoring history (English)
│   ├── RuidoVisual-midi-mapping-standard.json # Standard MIDI mapping (loadable)
│   ├── smoke-0.3.8-progreso.md    # Browser validation log, step by step
│   ├── plans/                     # Work plans: `refactors/` and `desarrollo/` (index in its README)
│   ├── auditoria/                 # Published code audit reports (primary source)
│   ├── filosofia/                 # Development philosophy (living document)
│   └── capturas/                  # Manual screenshots (added manually)
├── typo/
│   ├── fonts.json          # TYPO/ASCII font manifest (optional)
│   └── (fonts .ttf/.otf/.woff/.woff2, also in subfolders with licenses; not versioned — loaded at runtime)
└── js/                     # ES modules (vanilla JS, no dependencies)
    ├── vj-app.js               # Entry point / orchestrator
    ├── utils.js                # Shared utilities (sanitization, mirror mapping)
    ├── blend-modes.js          # Blend mode definitions (shader-based, active since 0.3.1)
    ├── effect-registry.js      # Single source of truth: the effect catalog
    ├── font-loader.js          # TYPO/ASCII font loading (FontFace API)
    ├── text-renderer.js        # TYPO text and ASCII atlas rendering on 2D canvas
    ├── video-layer.js          # Video layer (file/camera + texture)
    ├── layer-manager.js        # Management of N dynamic layers
    ├── layer-effects.js        # Per-layer effect state
    ├── midi-controller.js      # Hardware MIDI support (fixed-layer mapping)
    ├── fader-automation.js     # Fader automation (loop/bounce)
    ├── motion/                 # Motion Control engine (per-layer keyframes)
    │   ├── motion-easing.js    # Easing curves (Linear/Ease-in/Ease-out/Smooth/Bounce)
    │   ├── motion-clip.js      # Pure clip model: keyframes, segments, JSON serialization
    │   └── motion-service.js   # REC sessions and per-layer playback (rAF)
    ├── project-manager.js      # Project save/load (effects + MIDI + mapping + audio + modulation)
    ├── recorder-manager.js     # Video/audio recording (MediaRecorder)
    ├── performance-monitor.js  # FPS metrics
    ├── projector-window.js     # Projection window logic
    ├── core/                   # Cross-cutting core: state, events, DI, windows
    │   ├── state-manager.js · events.js · service-container.js
    │   └── window-manager.js · window-protocol.js
    ├── services/               # Domain services
    │   ├── composition-service.js · layer-service.js · effect-service.js
    │   └── output-service.js · renderer-interface.js
    ├── renderer/               # Render backend (WebGL2 today)
    │   ├── webgl-renderer.js   # WebGL2 rendering + per-layer FBOs
    │   └── shader-library.js   # GLSL shaders (vertex + fragment)
    ├── ui/                     # UI facade and subcontrollers
    │   └── ui-controller.js + ui-*.js
    ├── audio/                  # Audio engine, window and modulation
    │   └── audio-controller.js · audio-window.js · audio-registry.js · modulation-*.js
    ├── transitions/            # Layer transitions (swap model)
    │   ├── transitions.js          # Pure catalog: types, patterns, easing, validation
    │   └── transition-service.js   # Orchestration: state, roulette, fade and swap
    ├── 3d/                     # 3D models (3D path, WebGPU/3D plan)
    │   └── glb-parser.js          # GLB/glTF 2.0 parser with no dependencies (phase A1)
    ├── guion/                  # Scripting: scene document and coverage pass
    │   ├── guion-model.js          # Pure model: scenes, sanitizing and versioned JSON
    │   ├── guion-service.js        # Transport: loads, applies, chains and measures
    │   ├── guion-applier.js        # One scene applied to the engine (via the facade)
    │   ├── guion-executor.js       # Full pass: caption, report and state hygiene
    │   ├── cobertura.js            # Coverage generators derived from the catalogues
    │   └── synthetic-source.js     # Test pattern generated in memory (layer source)

    └── mapping/                # Projection mapping subsystem
        ├── mapping-constants.js    # Shared constants (default regions)
        ├── mapping-editor.js       # Mapping broker: state + window + postMessage
        ├── mapping-editor-window.js # Visual editor in the separate window (viewer + SVG)
        └── mapping-renderer.js     # Mapping WebGL render (facade + polygons)

For a detailed description of each module's role, the data flow, and the mapping coordinate conventions, see documentation.md.

---

Additional documentation

• user_manual.md — User manual: setup, basic sequence, MIDI, mapping, recording, and troubleshooting.
• documentation.md — Detailed architecture, role of each module, rendering pipeline, mapping communication protocol, and coordinate conventions.
• docs/video_preparation.md — Video preparation guide with FFmpeg (bounce loops, recommended formats).
• docs/browser_configuration.md — Browser-specific configuration (per-window zoom, permissions).
• docs/refactor_history.md — Cumulative technical history of the project refactorings.
• docs/RuidoVisual-midi-mapping-standard.json — Standard MIDI mapping, loadable from the MIDI mapping modal ("Cargar Mapeo").
• manual_de_usuario.md — User manual in Spanish.

---

Video Preparation

For the bounce effect (forward and backward playback) it is worth pre-processing the videos with FFmpeg instead of resolving it live: the guide with the command, the recommended formats and the rest of the tasks is in docs/video_preparation.md. For the same thing without a command line, two free tools: OBS Studio (its virtual camera simulates a webcam, so you can load any window or broadcast signal as a layer) and HandBrake (encode videos without typing commands).

---

Released versions

Every tag in the repository is a complete snapshot of the project at a milestone: you can download it and run it exactly as it was at that time. The table of release versions, with its date and what each one includes, and the command to run a specific one, live in the CHANGELOG.

---

License

This project is licensed under GPL-3.0. See LICENSE (official text) and LICENSE.es (Spanish summary).

All shader code (GLSL), JavaScript modules, HTML, and CSS are distributed under the same license. The original author is Ruido Visual Lab.

---

AI-assisted development

RVWVP has been developed with an AI-assisted workflow using different systems over time:

• December 2025 – early 2026: project start (first commit: 2025-12-14) with ChatGPT.
• First half of 2026: Vercel v0 stage (UI generation); dropped when that API was no longer used.
• Mid 2026: DeepSeek LLM.
• Summer 2026 (transition marked by the .zcode/ cleanup, 2026-08-03): started using ZCode, with GLM 5.2 and DeepSeek Flash v4.
• September 2026 (most recent): also using the DeepSeek harness.

How the work flows: I hand-review all the code and additionally do a peer review by cross-checking it against other models to spot nuances. DeepSeek (web version) has continuously supervised the work: in every iteration it explains parts of the code to me, while ZCode (or whichever code generator is in turn) executes the designs I had previously produced with DeepSeek's occasional help.

The code tour (data/tour.json + @code-tour: comments) is precisely what makes that audit possible: it enables self-review and third-party review of the skeleton and architectural design of each part of the program, and it can serve as the basis for practical, scripted training courses.

The development philosophy —the story of how the project got here, how the work is split between human and machine, what can and cannot be demonstrated about that supervision, and how human validation is dosed— is written up in docs/filosofia/filosofia_de_desarrollo.md (in Spanish).

Intermediate dates are approximate; the verifiable anchors in the git history are the first commit (2025-12-14) and the .zcode/ cleanup (2026-08-03).

---

RVWVP · Ruido Visual Lab Web-VJ Project · GPL-3.0