Work
Rap Almanac — Client Project (Rap Research Labs)
Overview
Blacksky serves as the engineering partner for Rap Research Labs (RRL) on the Rap Almanac, a full-scale music intelligence platform. Blacksky designed and built the entire backend data pipeline, NLP enrichment system, and operator dashboard powering the platform.
Client: Rap Research Labs (RRL) Project: Rap Almanac — a hip-hop music intelligence database Engagement start: February 2026 (ongoing) Status: Active development
What Blacksky Built
Blacksky delivered a complete data pipeline and intelligence system across seven workstreams:
- Scraper — Genius API scraper with three-phase pipeline: artists → albums → songs. Incremental logic to avoid re-scraping existing records.
- Blacksky NLP — Full open-source replacement for IBM Watson NLU (see below)
- Rhyme Analysis — Integration with client-provided Java-based rhyme scheme detection
- Word2Vec — Incremental vector embeddings and semantic similarity search across the lyric corpus
- Geo — Location extraction, geocoding, and map visualization using MusicBrainz, Wikidata, and Nominatim
- Elasticsearch — Full-text search with faceted filtering as the final pipeline stage
- Sync — Local → Railway MySQL push pipeline with validation and diff checking
Blacksky NLP — IBM Watson Replacement
A key deliverable was replacing the client's IBM Watson NLU dependency with a fully open-source, cost-free NLP system. Blacksky built five modular analyzers:
Sentiment Analysis — DistilBERT model. Returns a score from -1.0 (negative) to +1.0 (positive) plus a label. Handles long lyrics by chunking and averaging across chunks.
Emotion Detection — RoBERTa with GoEmotions fine-tuning. Returns Watson-compatible 5-axis output: sadness, joy, fear, disgust, and anger (each scored 0–1 per song).
Named Entity Recognition — spaCy large English model. Extracts people, places, organizations, and events from lyrics. Filters false positives common to rap (ad-libs, filler words).
Keyword Extraction — KeyBERT with Maximal Marginal Relevance for diversity. Returns top 20 keywords and phrases with relevance scores.
Category Classification — BART zero-shot classification against a 46-category taxonomy built specifically for hip-hop content, covering Society, Lifestyle, Geography, Personal, Art & Entertainment, Politics, and more. Returns up to 5 categories per song with confidence scores.
Platform Architecture
Stack: Python 3.12, Flask, MySQL, Elasticsearch (local) / OpenSearch (production), Docker, GitHub Actions CI/CD, Railway hosting, Java Lambda for rhyme detection
Database: MySQL (hhwc_data) with tables for sentiment, emotions, entities (people, places), keywords, and categories — all linked per song
Dashboard: 72 Flask routes covering Library search, Scraper control, NLP results, Geo visualization, Word2Vec search, Rhyme analysis, and Sync tools
Why This Project Matters for Blacksky
The Rap Almanac demonstrates Blacksky's ability to deliver production-grade data engineering and custom NLP at a fraction of the cost of enterprise solutions like Watson. It also reflects Blacksky's identity — this is a hip-hop intelligence platform built by people who live in that culture.