Back
gh

ptnghia-j/ChordMiniApp: Music Analysis, Chord Recognition, Beat Tracking, Guitar Diagrams, Piano Visualizer, Lyrics Transcription Application, context-aware LLM inference for analysis from uploaded audio and YouTube video

Music Analysis, Chord Recognition, Beat Tracking, Guitar Diagrams, Piano Visualizer, Lyrics Transcription Application, context-aware LLM inference for analysis from uploaded audio and YouTube video - ptnghia-j/ChordMiniApp

by ptnghia-j github.com 1,334 words
View original

ChordMini

Open-source music analysis tool for chord recognition, beat tracking, piano visualizer, guitar diagrams, and lyrics synchronization.

Features Overview

ChordMini Homepage Light ChordMini Homepage Dark

Clean, intuitive interface for YouTube search, URL input, and recent video access.

🎵 Beat & Chord Analysis

Beat Chord Grid Beat Chord Grid with Lyrics

Chord progression visualization with synchronized beat detection and grid layout with add-on features: Roman Numeral Analysis, Key Modulation Signals, Simplified Chord Notation, Enhanced Chord Correction, and song segmentation overlays for structural sections like intro, verse, chorus, bridge, and outro.

🎵 Guitar Diagrams

Guitar Diagrams

Interactive guitar chord diagrams with accurate fingering patterns from the official @tombatossals/chords-db database, featuring multiple chord positions, synchronized beat grid integration, and exact slash-chord matching when the database includes a dedicated inversion shape.

🎹 Piano Visualizer

Piano Visualizer

Real-time piano roll visualization with falling MIDI notes synchronized to chord playback. Features a scrolling chord strip, interactive keyboard highlighting, smoother playback-synced rendering, segmentation-aware dynamics shaping, and MIDI file export for importing chord progressions into any DAW.

🎤 Lead Sheet with AI Assistant

Lead Sheet with AI

Synchronized lyrics transcription with AI chatbot for contextual music analysis and translation support.

🚀 Quick Setup

Prerequisites

Setup Steps

  1. Clone and install Clone with submodules in one command (for fresh clones) 
    git lfs install
git clone --recursive https://github.com/ptnghia-j/ChordMiniApp.git
cd ChordMiniApp
git lfs pull
npm install

    If you already cloned the repo before SongFormer was added

    git pull
git lfs pull
    git lfs pull downloads the large SongFormer model files referenced by this repo, including the checkpoint binaries stored as Git LFS objects.

    Verify that submodules are populated

    ls -la python_backend/models/Beat-Transformer/
ls -la python_backend/models/Chord-CNN-LSTM/
ls -la python_backend/models/ChordMini/

    If chord recognition encounters issue with fluidsynth:

    Install FluidSynth for MIDI synthesis 
    # --- Windows ---
choco install fluidsynth
# --- macOS ---
brew install fluidsynth
# --- Linux (Debian/Ubuntu-based) ---
sudo apt update
sudo apt install fluidsynth
  2. Environment setup 
    cp .env.example .env.local
    Edit .env.local: 
    NEXT_PUBLIC_PYTHON_API_URL=http://localhost:5001
NEXT_PUBLIC_FIREBASE_API_KEY=your_firebase_api_key
NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN=your_project.firebaseapp.com
NEXT_PUBLIC_FIREBASE_PROJECT_ID=your_project_id
NEXT_PUBLIC_FIREBASE_STORAGE_BUCKET=your_project.appspot.com
NEXT_PUBLIC_FIREBASE_MESSAGING_SENDER_ID=your_sender_id
NEXT_PUBLIC_FIREBASE_APP_ID=your_app_id
  3. Start Python backend (Terminal 1) 
    cd python_backend
python -m venv myenv
source myenv/bin/activate  # On Windows: myenv\Scripts\activate
pip install -r requirements.txt
python app.py
  4. Start frontend (Terminal 2) 
    npm run dev
  5. Open application Visit http://localhost:3000

Prerequisites

Quick Start

  1. Download configuration files 
    curl -O https://raw.githubusercontent.com/ptnghia-j/ChordMiniApp/main/docker-compose.prod.yml
curl -O https://raw.githubusercontent.com/ptnghia-j/ChordMiniApp/main/.env.docker.example
  2. Configure environment 
    cp .env.docker.example .env.docker
# Edit .env.docker with your API keys (see API Keys Setup section below)
  3. Start the application 
    docker compose -f docker-compose.prod.yml --env-file .env.docker up -d
  4. Access the application Visit http://localhost:3000
  5. Stop the application 
    docker compose -f docker-compose.prod.yml down

Note: If you have Docker Compose V1 installed, use docker-compose (with hyphen) instead of docker compose (with space).

Docker Desktop GUI (Alternative)

If you prefer using Docker Desktop GUI:

  1. Open Docker Desktop
  2. Go to “Images” tab and search for ptnghia/chordminiapp-frontend and ptnghia/chordminiapp-backend
  3. Pull both images
  4. Use the “Containers” tab to manage running containers

Required Environment Variables

Edit .env.docker with these required values:

See the API Keys Setup section below for detailed instructions on obtaining these keys.

📋 Detailed Setup Instructions

Firebase Setup

  1. Create Firebase project
  2. Enable Firestore Database
    • Go to “Firestore Database” in the sidebar
      • Click “Create database”
      • Choose “Start in test mode” for development
  3. Get Firebase configuration
    • Go to Project Settings (gear icon)
      • Scroll down to “Your apps”
      • Click “Add app” → Web app
      • Copy the configuration values to your .env.local
  4. Create Firestore collections The app uses the following Firestore collections. They are created automatically on first write (no manual creation required):
    • transcriptions — Beat and chord analysis results (docId: ${videoId}_${beatModel}_${chordModel})
      • translations — Lyrics translation cache (docId: cacheKey based on content hash)
      • lyrics — Music.ai transcription results (docId: videoId)
      • keyDetections — Musical key analysis cache (docId: cacheKey)
      • audioFiles — Audio file metadata and URLs (docId: videoId)
      • segmentationJobs — Async SongFormer segmentation jobs and persisted results (docId: seg_<timestamp>_<uuid>)
  5. Enable Anonymous Authentication
    • In Firebase Console: Authentication → Sign-in method → enable Anonymous
  6. Configure Firebase Storage
    • Set environment variable: NEXT_PUBLIC_FIREBASE_STORAGE_BUCKET=your_project_id.appspot.com
      • Folder structure:
      • audio/ for audio files - video/ for optional video files
      • Filename pattern requirement: filenames must include the 11-character YouTube video ID in brackets, e.g. audio_[VIDEOID]_timestamp.mp3 (enforced by Storage rules)
      • File size limits (enforced by Storage rules):
      • Audio: up to 50MB - Video: up to 100MB

API Keys Setup

Music.ai API (deprecated - MUSIC.ai no longer provide individual API key, only business plan)

# 1. Sign up at music.ai
# 2. Get API key from dashboard
# 3. Add to .env.local
NEXT_PUBLIC_MUSIC_AI_API_KEY=your_key_here

Google Gemini API

# 1. Visit Google AI Studio
# 2. Generate API key
# 3. Add to .env.local
NEXT_PUBLIC_GEMINI_API_KEY=your_key_here

🏗️ Backend Architecture

ChordMiniApp uses a hybrid backend architecture:

🔧 Local Development Backend (Required)

For local development, you must run the Python backend on localhost:5001:

☁️ Production Backend (your VPS)

Production deployments is configured based on your VPS and url should be set in the NEXT_PUBLIC_PYTHON_API_URL environment variable.

Prerequisites

Quick Setup

  1. Navigate to backend directory 
    cd python_backend
  2. Create virtual environment 
    python -m venv myenv
# Activate virtual environment
# On macOS/Linux:
source myenv/bin/activate
# On Windows:
myenv\Scripts\activate
  3. Install dependencies 
    pip install --no-cache-dir Cython>=0.29.0 numpy==1.22.4
pip install --no-cache-dir madmom>=0.16.1
pip install --no-cache-dir -r requirements.txt
    In cases of conflict with spleeter, httpx, use —no-deps to skip installing dependencies of spleeter.
  4. Start local backend on port 5001 
    python app.py
    The backend will start on http://localhost:5001 and should display: 
    Starting Flask app on port 5001
App is ready to serve requests
Note: Using port 5001 to avoid conflict with macOS AirPlay/AirTunes on port 5000
  5. Verify backend is running Open a new terminal and test the backend: 
    curl http://localhost:5001/health
# Should return: {"status": "healthy"}
  6. Start frontend development server 
    # In the main project directory (new terminal)
npm run dev
    The frontend will automatically connect to http://localhost:5001 based on your .env.local configuration.

Backend Features Available Locally

Environment Variables for Local Backend

Create a .env file in the python_backend directory:

# Optional: Redis URL for distributed rate limiting
REDIS_URL=redis://localhost:6379

# Optional: Genius API for lyrics
GENIUS_ACCESS_TOKEN=your_genius_token

# Flask configuration
FLASK_MAX_CONTENT_LENGTH_MB=150
CORS_ORIGINS=http://localhost:3000,http://127.0.0.1:3000

Troubleshooting Local Backend

Backend connectivity issues:

# 1. Verify backend is running
curl http://localhost:5001/health
# Expected: {"status": "healthy"}

# 2. Check if port 5001 is in use
lsof -i :5001  # macOS/Linux
netstat -ano | findstr :5001  # Windows

# 3. Verify environment configuration
cat .env.local | grep PYTHON_API_URL
# Expected: NEXT_PUBLIC_PYTHON_API_URL=http://localhost:5001

# 4. Check for macOS AirTunes conflict (if using port 5000)
curl -I http://localhost:5000/health
# If you see "Server: AirTunes", that's the conflict we're avoiding

Frontend connection errors:

# Check browser console for errors like:
# "Failed to fetch" or "Network Error"
# This usually means the backend is not running on port 5001

# Restart both frontend and backend:
# Terminal 1 (Backend):
cd python_backend && python app.py

# Terminal 2 (Frontend):
npm run dev

Import errors:

# Ensure virtual environment is activated
source myenv/bin/activate  # macOS/Linux
myenv\Scripts\activate     # Windows

# Reinstall dependencies
pip install -r requirements.txt

External APIs & Services

We sincerely thank the following APIs and services for their support and contribution to the project.