Quartz 4

report

5 min read

SoundCloud Stats Dashboard

ESIEE Paris E4 DSIA DevOps Data Final Project

Video Demo

A music stats dashboard where users log in with their SoundCloud account to see their tracks, playlists, liked tracks, and top artists. Users can toggle their profile to public so that other visitors can browse their stats.

Architecture

The app has three components: a frontend, a backend, and a PostgreSQL database.

The frontend is static HTML/CSS/JS served by Nginx. There’s no build step or framework. It calls the backend API through relative URLs (/api/me, /api/users, etc.). Nginx proxies anything under /api/ and /health to the backend.

The backend is a Node.js Express server (port 3000). It handles SoundCloud OAuth login (PKCE flow), fetches user data from the SoundCloud API, and stores it in PostgreSQL. Fetched data (tracks, playlists, likes) is cached in a stats_snapshots table so it can be served again without hitting SoundCloud — this is also how public profiles work. Routes under /api/me and /api/preferences require an active session; everything else is public.

PostgreSQL stores four tables:

  • users — SoundCloud profiles
  • tokens — OAuth access/refresh tokens
  • stats_snapshots — cached tracks, playlists, likes, top artists
  • preferences — per-user settings (public profile toggle)

The schema is applied automatically on first startup by mounting schema.sql into PostgreSQL’s init directory. Data persists via a Docker volume (docker-compose) or a PersistentVolumeClaim (Kubernetes).

The stack can run locally with docker-compose (port 8080) or on Kubernetes with Minikube. In both cases the three containers communicate over an internal network. The Kubernetes deployment exposes the app on localhost:8080 via a port-forward to the Ingress controller, so the same SoundCloud redirect URI works in both environments.

Tech Stack

  • Backend: Node.js 20, Express 5, pg
  • Frontend: Static HTML/CSS/JS, Nginx
  • Database: PostgreSQL 15
  • Containers: Docker
  • Orchestration: Kubernetes (Minikube)
  • CI/CD: GitHub Actions
  • Container Registry: GHCR (GitHub Container Registry)

Installation

Prerequisites

  • Git
  • Docker and Docker Compose
  • Node.js 20 (only needed to run tests locally)
  • Minikube and kubectl (only needed for Kubernetes deployment)

Clone and configure

git clone https://github.com/melchiorlaurens/devops-esiee-proj-labs.git
cd devops-esiee-proj-labs/project
cp .env.example .env

Edit .env and fill in the values. The file looks like this:

POSTGRES_PASSWORD=esieedevops5567
SESSION_SECRET=dev-session-secret
SOUNDCLOUD_CLIENT_ID=your_soundcloud_client_id
SOUNDCLOUD_CLIENT_SECRET=your_soundcloud_client_secret
SOUNDCLOUD_REDIRECT_URI=http://localhost:8080/api/auth/callback

SOUNDCLOUD_CLIENT_ID and SOUNDCLOUD_CLIENT_SECRET come from the SoundCloud developer portal. The redirect URI must match what is configured in the SoundCloud app settings.

The database connection variables (DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASSWORD) are set automatically by docker-compose and by the Kubernetes manifests — they don’t need to be in .env.

Run with docker-compose

docker-compose up --build

This starts postgres, backend, and frontend. The database schema is created automatically on first run. The app is available at http://localhost:8080.

To stop: docker-compose down. To stop and wipe the database: docker-compose down -v.

Deploy on Kubernetes (Minikube)

Start the cluster and enable ingress:

minikube start
minikube addons enable ingress

Apply the manifests in order:

kubectl apply -f k8s/base/namespace.yaml
kubectl apply -f k8s/base/schema-configmap.yaml
kubectl apply -f k8s/base/postgres.yaml
kubectl apply -f k8s/base/backend.yaml
kubectl apply -f k8s/base/frontend.yaml
kubectl apply -f k8s/base/ingress.yaml

postgres.yaml includes the postgres-secret resource, so postgres-secret.yaml does not need to be applied separately.

Wait for pods to be ready:

kubectl get pods -n app -w

Expose the Ingress controller on port 8080:

kubectl port-forward -n ingress-nginx svc/ingress-nginx-controller 8080:80

Leave this running in its own terminal. The app is available at http://localhost:8080. To verify:

curl http://localhost:8080/api/health
# {"status":"ok","database":"connected"}

This uses the same localhost:8080 origin as docker-compose, so the SoundCloud OAuth redirect URI (http://localhost:8080/api/auth/callback) works without changes.

To check logs:

kubectl logs -n app deployment/postgres
kubectl logs -n app deployment/backend
kubectl logs -n app deployment/frontend

The Kubernetes manifests pull images from GHCR (ghcr.io/melchiorlaurens/devops-esiee-proj-labs/backend:latest and frontend:latest), built and pushed by the CI/CD pipeline.

Tests

The backend has integration tests (Jest + Supertest) that run against a real PostgreSQL database with no mocking.

What they cover:

  • /health returns 200 and confirms the database is connected
  • /api/hello returns a response
  • Protected routes (/api/me, /api/preferences) return 401 without a session
  • Full CRUD cycle: insert a user, check public listing is empty, set profile to public, verify user appears, toggle back to private, delete user and verify cascade

To run locally, a running PostgreSQL instance is required. The connection is configured via environment variables:

cd project/backend
npm ci
DB_HOST=localhost DB_PORT=5432 DB_NAME=testdb DB_USER=postgres DB_PASSWORD=esieedevops5567 npm test

Adjust DB_PASSWORD to match the local PostgreSQL instance. In CI, a dedicated PostgreSQL service container is created automatically with its own credentials.

CI/CD Pipeline

Defined in .github/workflows/ci.yml. Runs on every push or pull request to main.

Test job — Spins up a PostgreSQL 16 service container, installs dependencies with npm ci, and runs npm test. If any test fails, the pipeline stops and the build job is skipped.

Build and push job — Builds Docker images for backend and frontend, and pushes them to GHCR. Each image is tagged with the git SHA and latest. On pull requests, images are built but not pushed (only the build is verified).

After new images are pushed, Kubernetes pods can be updated with:

kubectl rollout restart deployment/backend -n app
kubectl rollout restart deployment/frontend -n app

The manifests use imagePullPolicy: Always, so restarted pods pull the latest image.

API Routes

Public routes:

  • GET /health — database connectivity status
  • GET /api/hello — test endpoint
  • GET /api/auth/login — redirects to SoundCloud OAuth
  • GET /api/auth/callback — handles OAuth callback, creates user and session
  • GET /api/users — lists users with public profiles
  • GET /api/users/:id/tracks — public user’s cached tracks
  • GET /api/users/:id/playlists — public user’s cached playlists
  • GET /api/users/:id/top-artists — public user’s top artists
  • GET /api/users/:id/top-tracks — public user’s top tracks

Protected routes (401 without session):

  • GET /api/me — logged-in user’s profile
  • GET /api/me/tracks — user’s tracks from SoundCloud (cached)
  • GET /api/me/playlists — user’s playlists (cached)
  • GET /api/me/likes — user’s liked tracks (cached)
  • GET /api/me/top-artists — top artists computed from tracks/likes
  • GET /api/me/top-tracks — top tracks sorted by play count
  • GET /api/preferences — user preferences
  • PUT /api/preferences — update preferences (e.g., public profile toggle)
  • POST /api/auth/logout — destroys session

Graph View