Curated IPTV playlist + EPG tooling

Builds a validated IPTV playlist from iptv-org/iptv and a matching XMLTV guide
from iptv-org/epg (batched to bound memory), then publishes both to
chillidonut.com/junk/iptv/ via gsutil. Outputs and clones are not committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

.gitignore

@@ -0,0 +1,5 @@
+# build outputs — regenerated by refresh.ts, published via deploy.ts (not committed)
+/iptv.m3u
+/guide.xml
+/iptv-build-report.txt
+/build.log

README.md

@@ -0,0 +1,102 @@
+# IPTV — curated playlist + EPG
+
+Tooling that builds a personal, validated IPTV playlist from
+[iptv-org/iptv](https://github.com/iptv-org/iptv) plus a matching XMLTV guide from
+[iptv-org/epg](https://github.com/iptv-org/epg), then publishes the two outputs to
+`chillidonut.com/junk/iptv/` (served to **TiviMate**).
+
+This repo is **tooling only**. The build outputs (`iptv.m3u`, `guide.xml`), the
+upstream clones, and `node_modules` are not committed — clones live in `/tmp`.
+
+## Files
+| File | What it does |
+|------|--------------|
+| `build-iptv.mjs` | Curate + validate playlist → `iptv.m3u` |
+| `make-epg.mjs` | Bind `tvg-id`s to epg channels → rewrites `iptv.m3u`, writes `custom.channels.xml` |
+| `grab-epg.ts` | Batched, memory-bounded EPG grab → `guide.xml` |
+| `refresh.ts` | Bun orchestrator: build → bind → grab |
+| `deploy.ts` | Upload `iptv.m3u` + `guide.xml` to the GCS bucket (gzipped) |
+
+## Usage
+```sh
+bun refresh.ts              # build everything → iptv.m3u + guide.xml
+bun refresh.ts --revalidate # also re-probe every stream from scratch
+bun deploy.ts               # publish the two outputs to chillidonut.com/junk/iptv/
+
+# individual steps:
+bun build-iptv.mjs          # playlist only (--dry = counts only, no probing)
+bun make-epg.mjs            # bind EPG ids (needs the epg clone in /tmp)
+bun grab-epg.ts             # grab guide.xml (env: BATCH, DAYS, MAXCONN, LIMIT)
+```
+Requires [Bun](https://bun.com); `deploy.ts` also needs `gsutil`. Upstream clones +
+the grabber's `node_modules` are created on demand under `/tmp/iptv-org-build/` —
+throwaway, so a reboot (which clears `/tmp`) just means the next run re-bootstraps.
+
+### TiviMate
+Add `https://chillidonut.com/junk/iptv/iptv.m3u` as a playlist. TiviMate reads
+`group-title` as categories and binds the guide via the `url-tvg` header + `tvg-id`s.
+
+## How it works
+1. **Curate** (`build-iptv.mjs`): from a clone of iptv-org/iptv `streams/*.m3u`,
+   select sections — AU + AU Samsung, US broadcast + Pluto/Samsung/Tubi/Xumo,
+   flagship/world News, Comedy Central, Español (Mexico + Spanish news), AR/CO/CL
+   terrestrial, bespoke European terrestrial. Curated sections use name whitelists;
+   name collisions get a source tag, e.g. `Comedy Central (US)` vs `(DK)`.
+2. **Validate**: every stream is probed (HTTP, from this machine) and dead/geo-blocked
+   ones dropped. Cached in `/tmp/iptv-org-build/validation.json` (fast re-runs;
+   `--revalidate` ignores it). Probing from AU means US-geo-locked feeds fall away.
+3. **Bind EPG** (`make-epg.mjs`): assigns a canonical `tvg-id` per channel — by base
+   id (ignoring iptv-org's `@feed` suffix), and for **Pluto** by the `plu-<id>` in the
+   URL, which *is* epg's `pluto.tv` `site_id` (so blank-`tvg-id` Pluto channels still
+   get a guide). Writes `custom.channels.xml` (one source per id).
+4. **Grab** (`grab-epg.ts`): runs iptv-org/epg's grabber over `custom.channels.xml`,
+   batched, → `guide.xml`. See the memory note below.
+
+### EPG coverage / freshness
+~1,000 of ~2,000 channels are guide-bound. The rest have no usable id: many
+Samsung/Tubi/Xumo FAST channels ship a blank `tvg-id`, and iptv-org/epg has **no
+Samsung or Tubi site**. XMLTV does **not** loop — programmes have absolute times, so
+the guide goes stale (FAST ~12–48 h, broadcasters ~3–7 days) and must be regenerated
+on a schedule (daily). A daily `bun refresh.ts && bun deploy.ts` job is the intended
+setup (not yet wired up — see memory note).
+
+## Deploying
+`bun deploy.ts` uploads only `iptv.m3u` + `guide.xml` to `gs://chillidonut.com/junk/iptv/`:
+- **`cp`, not `rsync`** — it adds/overwrites just these two objects; everything else
+  in the bucket's `junk/` is left untouched (never synced or deleted).
+- **`-Z`** gzips on upload and sets `Content-Encoding: gzip` (~32 MB guide → ~3–4 MB;
+  TiviMate decompresses transparently). `url-tvg` stays `…/guide.xml` — don't pre-gzip,
+  that would double-compress.
+
+(On chillidonut.com, `static/junk/**` is excluded from `hugo deploy`, which is why
+these go straight to the bucket instead of through the site build.)
+
+## Known limitations / notes for later
+
+### ⚠️ The EPG grab is very memory-hungry
+The iptv-org/epg grabber loads the **entire** iptv-org API dataset into memory and
+retains all fetched programme data until it serializes at the end. For ~1,000
+channels a single process climbs to **~20 GB+**.
+
+`grab-epg.ts` works around this by splitting into batches that each run as a
+**separate `bun` process** (memory reclaimed when each exits). That bounds the
+*sustained* footprint but **not** the per-batch peak: the baseline API load alone is
+several GB, and some sites retain a lot, so individual batches have spiked to **~18 GB**
+even at `BATCH=25`. Programme volume is tiny (≤~2k/batch), so this is grabber overhead,
+not our data — shrinking `BATCH` barely helps.
+
+**Not yet suitable for an unattended daily job on a working machine.** Ideas to make it
+resource-efficient (none implemented — batching is a stopgap):
+- **Trim the API load**: pre-filter so the grabber loads only our channels' data, not
+  the whole iptv-org dataset (kills the multi-GB baseline).
+- **Stream to disk**: fork the grab to write each channel's programmes incrementally
+  and release them, instead of one giant in-memory guide.
+- **Run it off-machine**: do the grab in CI/cloud (iptv-org/epg itself runs as a daily
+  GitHub Action) and just publish the resulting `guide.xml`.
+- **Don't self-grab**: point `url-tvg` at a maintained hosted XMLTV for channels that
+  have one, and only self-grab the gaps.
+
+### Other
+- Samsung/Tubi (and most blank-`tvg-id` FAST channels) have no EPG — see above.
+- Validation reflects *this machine's* connectivity (currently AU). Re-run with
+  `--revalidate` from a VPN to re-evaluate US-geo feeds.

build-iptv.mjs

@@ -0,0 +1,312 @@
+// Curated IPTV playlist builder — sources from a clone of iptv-org/iptv.
+// The clone + validation cache live in /tmp (portable, throwaway — nothing lands
+// in the repo). Writes iptv.m3u next to this script; deploy.ts publishes it to GCS.
+//
+//   node build-iptv.mjs --dry          assemble + counts only, no validation
+//   node build-iptv.mjs                assemble + validate (cached) + write iptv.m3u
+//   node build-iptv.mjs --revalidate   re-probe every stream, ignoring the cache
+//
+// Update upstream first with: git -C /tmp/iptv-org-build/iptv-org-repo pull
+import { readFileSync, writeFileSync, readdirSync, existsSync, mkdirSync } from 'node:fs';
+import { execSync } from 'node:child_process';
+import { fileURLToPath } from 'node:url';
+import path from 'node:path';
+
+const HERE = path.dirname(fileURLToPath(import.meta.url));
+const TMP = '/tmp/iptv-org-build';            // clone + probe cache: portable, throwaway
+const REPO = path.join(TMP, 'iptv-org-repo');
+const STREAMS = path.join(REPO, 'streams');
+const VCACHE = path.join(TMP, 'validation.json');
+const OUT = path.join(HERE, 'iptv.m3u');
+const REPORT = path.join(HERE, 'iptv-build-report.txt');
+const RAW = 'https://raw.githubusercontent.com/iptv-org/iptv/master/streams';
+const DRY = process.argv.includes('--dry');
+const REVALIDATE = process.argv.includes('--revalidate');
+const CONCURRENCY = 50;
+const TIMEOUT_MS = 7000;
+
+// ---------- bootstrap upstream clone ----------
+if (!existsSync(STREAMS)) {
+  mkdirSync(TMP, { recursive: true });
+  process.stderr.write(`Cloning iptv-org/iptv (shallow) into ${TMP} ...\n`);
+  execSync(`git clone --depth 1 https://github.com/iptv-org/iptv.git "${REPO}"`, { stdio: 'inherit' });
+}
+
+// ---------- parsing ----------
+const allFiles = readdirSync(STREAMS).filter((f) => f.endsWith('.m3u'));
+const cache = {};
+function parse(file) {
+  if (cache[file]) return cache[file];
+  let text;
+  try { text = readFileSync(path.join(STREAMS, file), 'utf8'); }
+  catch { return (cache[file] = []); }
+  const lines = text.split('\n');
+  const out = [];
+  for (let i = 0; i < lines.length; i++) {
+    if (!lines[i].startsWith('#EXTINF')) continue;
+    const extinf = lines[i];
+    const comma = extinf.indexOf(',');
+    const name = extinf.slice(comma + 1).trim();
+    let url = '';
+    for (let j = i + 1; j < lines.length; j++) {
+      const t = lines[j].trim();
+      if (!t) continue;
+      if (t.startsWith('#')) { if (t.startsWith('#EXTINF')) break; continue; }
+      url = t; break;
+    }
+    if (!url) continue;
+    const m = name.match(/\((\d+)p\)/);
+    out.push({ name, url, extinf, sourceFile: file, quality: m ? Number(m[1]) : 0,
+      geo: /\[Geo-blocked\]/i.test(name) });
+  }
+  return (cache[file] = out);
+}
+const everyEntry = () => allFiles.flatMap(parse);
+const filesEntries = (files) => files.flatMap(parse);
+
+// ---------- curated matchers ----------
+const NEWS = [
+  /^BBC News/i, /^Al Jazeera English/i, /^France 24( \(|$)/i, /^France 24 English/i,
+  /^DW( \(|$)/i, /^DW English/i, /^Sky News( \(|$)/i, /^Sky News Australia/i,
+  /^Euronews( \(|$)/i, /^Euronews English/i, /^CNA( \(|$)/i, /^Channel News ?Asia/i,
+  /^TRT World/i, /^Africanews( \(|$)/i, /^Africanews English/i, /^GB News/i, /^NHK World/i,
+  /^CGTN( \(|$)/i, /^CGTN Documentary/i, /^WION/i, /^Arirang TV( \(|$)/i, /^Arirang( \(|$)/i,
+  /^i24NEWS/i, /^TaiwanPlus/i, /^Bloomberg (Television|Originals|TV US|TV Australia|TV Plus|TV\+)/i,
+  /^Bloomberg TV( \(|$)/i, /^CNN International/i, /^CNN( \(|$)/i, /^ABC News Live( \(|\[|$)/i,
+  /^ABC News( \(|$)/i, /^NBC News Now/i, /^CBS News( \(|$|\[)/i, /^Scripps News/i,
+  /^LiveNOW from FOX/i, /^CBC News( \(|$)/i, /^CBC News Network/i, /^Newsmax/i, /^Global News National/i,
+];
+const NEWS_EXCLUDE = /Pashto|Arabic|Mubasher|Русск|Fran[cç]ais|Cha[iî]ne|Portugu|Deutsch|Espa[nñ]ol|Hebrew|Radio|Hindi|Urdu|Bangla|Mongolia|Bulgaria/i;
+const ESP_NEWS = [
+  /CNN en Espa/i, /France 24.*Espa/i, /^DW Espa/i, /Euronews.*Espa/i, /CGTN Espa/i,
+  /Deutsche Welle.*Espa/i, /Telemundo/i, /Univision/i, /TUDN/i, /Al Jazeera.*Espa/i,
+];
+// AR/CO/CL terrestrial & notable — Latin immersion (skips Comedy Central / flagship news already covered)
+const LATAM = [
+  /^Telefe/i, /^El Trece/i, /^El Nueve/i, /^TV Publica/i, /^Am[ée]rica TV/i, /^A24/i,
+  /^Bravo TV/i, /^Garage TV/i, /^Canal 9 /i,
+  /^Caracol/i, /RCN/i, /^Canal 1 /i, /^Se[ñn]al Colombia/i, /^Canal Capital/i,
+  /^Canal Institucional/i, /^NTN24/i, /^Teleantioquia/i, /^Telecaf/i, /^Telecaribe/i,
+  /^Telemedell/i, /^Cosmovision/i,
+  /^TVN( |3|$)/i, /^Mega( |noticias|$)/i, /^Canal 13/i, /^La Red/i, /^TV\+/i, /^UCV TV/i,
+];
+// Bespoke European terrestrial / region-defining (no Comedy Central, no flagship news dupes)
+const EUROPE = [
+  /^France 2( \(|$)/i, /^France 3( |\(|$)/i, /^France 4/i, /^France 5( \(|$)/i, /^Gulli/i, /^BFM TV/i, /^Arte/i,
+  /^Das Erste/i, /^ProSieben/i, /^3sat/i, /^Tagesschau 24/i, /^WDR Fernsehen( \(|$)/i,
+  /^NDR Fernsehen International/i, /^ZDF( \(|$)/i, /^ZDFneo/i,
+  /^Rai 1 HD/i, /^Rai 2 HD/i, /^Rai 3( \(|$)/i, /^Rai 4( \(|$)/i, /^Rai News/i, /^Rai Storia/i, /^Rai Movie/i, /^La7/i,
+  /^Canale 5/i, /^Italia 1/i, /^Rete 4/i,
+  /^La 1( \(|$)/i, /^Antena 3 Internacional/i, /^Canal Sur Andaluc/i, /^ETB ?[12] On/i,
+  /^BBC One( \(|$)/i, /^BBC Two( \(|$)/i, /^BBC Four/i, /^Channel 4( \(|$)/i, /^Channel 5( \(|$)/i, /^Film4/i,
+];
+const isNews = (n) => NEWS.some((r) => r.test(n)) && !NEWS_EXCLUDE.test(n);
+const isComedyCentral = (n) => /comedy central/i.test(n);
+const isEspNews = (n) => ESP_NEWS.some((r) => r.test(n));
+const isLatam = (n) => LATAM.some((r) => r.test(n));
+const isEurope = (n) => EUROPE.some((r) => r.test(n));
+
+// Streams confirmed good by hand (e.g. via ffprobe) that the HTTP probe flaps on.
+const FORCE_KEEP = [/^Comedy Central \(720p\)/i];
+
+// ---------- section assembly (priority order; first to claim a URL wins) ----------
+const seen = new Set();
+const sections = [];
+function add(meta, entries) {
+  const kept = [];
+  for (const e of entries) {
+    if (seen.has(e.url)) continue;
+    seen.add(e.url);
+    kept.push(e);
+  }
+  sections.push({ ...meta, entries: kept });
+}
+
+add({ title: 'News — flagships & world', group: 'News', dedupeByName: true,
+      sources: ['(cross-file pattern match across all country files)'],
+      filter: 'name matches flagship/major-news whitelist' },
+    everyEntry().filter((e) => isNews(e.name)));
+
+add({ title: 'Comedy Central', group: 'Comedy', dedupeByName: false,
+      sources: ['(cross-file pattern match across all country files)'],
+      filter: 'name contains "Comedy Central"' },
+    everyEntry().filter((e) => isComedyCentral(e.name)));
+
+add({ title: 'Espanol (Latin) — Mexico + Spanish news', group: 'Espanol', dedupeByName: false,
+      sources: ['mx.m3u', 'mx_pluto.m3u', 'mx_samsung.m3u', 'us_canelatv.m3u', '(+ Spanish-language news cross-file)'],
+      filter: 'all entries from MX/Canela files + Spanish-language news from any file' },
+    [...filesEntries(['mx.m3u', 'mx_pluto.m3u', 'mx_samsung.m3u', 'us_canelatv.m3u']),
+     ...everyEntry().filter((e) => isEspNews(e.name))]);
+
+add({ title: 'Latin America — AR/CO/CL terrestrial', group: 'LatAm', dedupeByName: false,
+      sources: ['ar.m3u', 'co.m3u', 'cl.m3u'],
+      filter: 'curated terrestrial/regional whitelist (Argentina, Colombia, Chile)' },
+    filesEntries(['ar.m3u', 'co.m3u', 'cl.m3u']).filter((e) => isLatam(e.name)));
+
+add({ title: 'Europe — terrestrial & regional', group: 'Europe', dedupeByName: false,
+      sources: ['fr.m3u', 'de.m3u', 'it.m3u', 'es.m3u', 'nl.m3u', 'uk.m3u'],
+      filter: 'curated iconic/terrestrial whitelist (excludes news & Comedy Central already covered)' },
+    filesEntries(['fr.m3u', 'de.m3u', 'it.m3u', 'es.m3u', 'nl.m3u', 'uk.m3u']).filter((e) => isEurope(e.name)));
+
+const bulk = [
+  ['Australia', 'AU', 'au.m3u'],
+  ['Australia — Samsung TV Plus', 'AU Samsung', 'au_samsung.m3u'],
+  ['US — Broadcast', 'US', 'us.m3u'],
+  ['US — Pluto TV', 'US Pluto', 'us_pluto.m3u'],
+  ['US — Samsung TV Plus', 'US Samsung', 'us_samsung.m3u'],
+  ['US — Tubi', 'US Tubi', 'us_tubi.m3u'],
+  ['US — Xumo', 'US Xumo', 'us_xumo.m3u'],
+];
+for (const [title, group, file] of bulk)
+  add({ title, group, dedupeByName: false, sources: [file], filter: 'all entries' }, parse(file));
+
+// ---------- validation (HTTP reachability, cached by URL) ----------
+async function check(url) {
+  const ctrl = new AbortController();
+  const timer = setTimeout(() => ctrl.abort(), TIMEOUT_MS);
+  try {
+    const res = await fetch(url, { method: 'GET', redirect: 'follow', signal: ctrl.signal,
+      headers: { 'User-Agent': 'VLC/3.0.20 LibVLC/3.0.20' } });
+    try { await res.body?.cancel(); } catch {}
+    return res.status;
+  } catch { return 0; }
+  finally { clearTimeout(timer); }
+}
+async function pool(items, n, fn) {
+  const ret = new Array(items.length);
+  let i = 0, done = 0;
+  async function worker() {
+    while (i < items.length) {
+      const idx = i++;
+      ret[idx] = await fn(items[idx]);
+      if (++done % 150 === 0) process.stderr.write(`  probed ${done}/${items.length}\n`);
+    }
+  }
+  await Promise.all(Array.from({ length: n }, worker));
+  return ret;
+}
+
+const allEntries = sections.flatMap((s) => s.entries);
+process.stderr.write(`Assembled ${allEntries.length} unique candidates across ${sections.length} sections.\n`);
+
+if (!DRY) {
+  let vcache = {};
+  if (!REVALIDATE) { try { vcache = JSON.parse(readFileSync(VCACHE, 'utf8')); } catch {} }
+  const toProbe = allEntries.filter((e) => !(e.url in vcache));
+  process.stderr.write(`Validating ${allEntries.length} streams — ${toProbe.length} to probe, ${allEntries.length - toProbe.length} cached.\n`);
+  const statuses = await pool(toProbe, CONCURRENCY, (e) => check(e.url));
+  toProbe.forEach((e, k) => { vcache[e.url] = statuses[k]; });
+  mkdirSync(TMP, { recursive: true });
+  writeFileSync(VCACHE, JSON.stringify(vcache));
+  for (const e of allEntries) {
+    const st = vcache[e.url] ?? 0;
+    e.status = st;
+    e.alive = (st >= 200 && st < 400) || FORCE_KEEP.some((r) => r.test(e.name));
+  }
+}
+
+// ---------- disambiguation ----------
+// When a channel name collides within a section, append a source tag (country,
+// then provider, then quality) — but only the dimensions that actually differ, so
+// single-source dupes don't get noise like "(AU)". Unique names are left untouched.
+function disambiguate(entries) {
+  const baseKey = (n) => n.toLowerCase()
+    .replace(/\(\d+p\)|\[[^\]]*\]|\((?:hevc|adaptive)\)/gi, '').replace(/\s+/g, ' ').trim();
+  const cc = (e) => (e.sourceFile.split(/[._]/)[0] || '').toUpperCase();
+  const prov = (e) => {
+    const p = e.sourceFile.replace(/\.m3u$/, '').split('_').slice(1).join(' ');
+    return p ? p.replace(/\b\w/g, (c) => c.toUpperCase()) : '';
+  };
+  const nameWith = (e, dims) => {
+    const tag = dims.map((d) => d(e)).filter(Boolean).join(' ');
+    return tag ? `${e.name} (${tag})` : e.name;
+  };
+  const groups = new Map();
+  for (const e of entries) {
+    const k = baseKey(e.name);
+    if (!groups.has(k)) groups.set(k, []);
+    groups.get(k).push(e);
+  }
+  for (const g of groups.values()) {
+    if (g.length < 2) continue;
+    // add the fewest source dimensions (country, then provider) needed to make
+    // names unique; quality already lives in the name so it's never re-appended.
+    const distinct = (dims) => new Set(g.map((e) => nameWith(e, dims))).size;
+    let active = [];
+    let best = distinct(active);
+    if (best < g.length) {
+      for (const d of [cc, prov]) {
+        const dn = distinct([...active, d]);
+        if (dn > best) { active = [...active, d]; best = dn; if (best === g.length) break; }
+      }
+    }
+    const counts = {};
+    for (const e of g) {
+      let n = nameWith(e, active);
+      counts[n] = (counts[n] || 0) + 1;
+      if (counts[n] > 1) n = `${n} (${counts[n]})`;
+      if (n !== e.name) e.displayName = n;
+    }
+  }
+}
+
+// ---------- post-process + write ----------
+const qrank = (e) => e.quality || 0;
+function finalize(s) {
+  let entries = DRY ? s.entries : s.entries.filter((e) => e.alive);
+  if (s.dedupeByName) {
+    const byName = new Map();
+    for (const e of entries) {
+      const key = e.name.toLowerCase().replace(/\(\d+p\)|\[[^\]]*\]/g, '').replace(/\s+/g, ' ').trim();
+      const cur = byName.get(key);
+      if (!cur || qrank(e) > qrank(cur)) byName.set(key, e);
+    }
+    entries = [...byName.values()];
+  }
+  entries.sort((a, b) => a.name.localeCompare(b.name));
+  return entries;
+}
+function extinfLine(e, group) {
+  const comma = e.extinf.indexOf(',');
+  const attrs = e.extinf.slice(0, comma).replace(/\s*group-title="[^"]*"/, '');
+  return `${attrs} group-title="${group}",${e.displayName ?? e.name}`;
+}
+
+const lines = ['#EXTM3U'];
+const reportLines = [];
+lines.push(
+  '# ============================================================',
+  '# Curated IPTV playlist — source: iptv-org/iptv',
+  '# Built by: build-iptv.mjs  (`bun refresh.ts` = playlist + EPG, `bun deploy.ts` = publish)',
+  '# Rebuild:  bun build-iptv.mjs            (cached, fast)',
+  '# Update:   git -C /tmp/iptv-org-build/iptv-org-repo pull && bun build-iptv.mjs --revalidate',
+  `# Validation: ${DRY ? 'SKIPPED (dry run)' : 'HTTP reachability probed from this machine; dead/geo-blocked dropped'}`,
+  '# Colliding names get a source tag, e.g. "Comedy Central (US)" vs "(DK)".',
+  '# Per-section comments below record upstream sources so this can be regenerated.',
+  '# ============================================================');
+
+let totalKept = 0;
+for (const s of sections) {
+  const finals = finalize(s);
+  disambiguate(finals);
+  totalKept += finals.length;
+  reportLines.push(`${s.title.padEnd(40)} kept ${String(finals.length).padStart(4)} / ${String(s.entries.length).padStart(4)} candidates`);
+  lines.push('', '# ------------------------------------------------------------',
+    `# SECTION: ${s.title}   (group-title="${s.group}")`,
+    ...s.sources.map((src) => src.startsWith('(') ? `# Upstream: ${src}` : `# Upstream: ${RAW}/${src}`),
+    `# Filter: ${s.filter}${s.dedupeByName ? ' + deduped by channel name (best quality)' : ''}`,
+    `# ${DRY ? 'Candidates' : 'Alive'}: ${finals.length}${DRY ? '' : ` / ${s.entries.length}`}`,
+    '# ------------------------------------------------------------');
+  for (const e of finals) {
+    if (/^(News|Comedy|Espanol|Latin|Europe)/.test(s.title)) lines.push(`# src: ${e.sourceFile}`);
+    lines.push(extinfLine(e, s.group), e.url);
+  }
+}
+
+mkdirSync(path.dirname(OUT), { recursive: true });
+writeFileSync(OUT, lines.join('\n') + '\n');
+const report = ['IPTV build report (' + (DRY ? 'DRY — not validated' : 'validated') + ')',
+  '='.repeat(62), ...reportLines, '-'.repeat(62),
+  `TOTAL kept: ${totalKept}`, `Output: ${OUT}`].join('\n');
+writeFileSync(REPORT, report + '\n');
+process.stderr.write('\n' + report + '\n');

deploy.ts

@@ -0,0 +1,18 @@
+#!/usr/bin/env bun
+// Publish the built playlist + guide to chillidonut.com's junk drawer on GCS.
+// Uses `cp` (NOT `rsync`) so it only adds/overwrites these two objects — anything
+// else already in the bucket's junk/ is left untouched (never synced or deleted).
+// `-Z` gzips on upload and sets Content-Encoding: gzip (served compressed on the
+// wire; ~32 MB guide -> ~3-4 MB). url-tvg stays …/guide.xml — do NOT pre-gzip.
+import { $ } from 'bun';
+import { existsSync } from 'node:fs';
+
+const HERE = import.meta.dir;
+const DEST = 'gs://chillidonut.com/junk/iptv/';
+const files = ['iptv.m3u', 'guide.xml'].map((f) => `${HERE}/${f}`);
+
+for (const f of files)
+  if (!existsSync(f)) { console.error(`missing ${f} — run \`bun refresh.ts\` first`); process.exit(1); }
+
+await $`gsutil -m -h ${'Cache-Control:public,max-age=3600'} cp -Z ${files} ${DEST}`;
+console.log(`uploaded ${files.length} files -> ${DEST}  (gzipped, Content-Encoding: gzip)`);

grab-epg.ts

@@ -0,0 +1,50 @@
+#!/usr/bin/env bun
+// Batched EPG grab — keeps memory bounded. The iptv-org grabber holds every
+// programme for every channel in RAM until the end (1000+ channels = many GB),
+// so we grab ~BATCH channels at a time, EACH IN ITS OWN bun process (memory is
+// reclaimed when the process exits), then merge the per-batch guides into one
+// XMLTV file. Tunable via env: BATCH, DAYS, MAXCONN, LIMIT, CHANNELS, OUT.
+import { $ } from 'bun';
+import { readFileSync, writeFileSync, mkdirSync, rmSync, existsSync } from 'node:fs';
+
+const HERE = import.meta.dir;
+const TMP = '/tmp/iptv-org-build';
+const EPG = `${TMP}/epg`;
+const CHANNELS = process.env.CHANNELS || `${TMP}/custom.channels.xml`;
+const OUT = process.env.OUT || `${HERE}/guide.xml`;
+const BATCH = Number(process.env.BATCH || 25);  // ~3 GB peak per batch (measured)
+const DAYS = Number(process.env.DAYS || 2);
+const MAXCONN = Number(process.env.MAXCONN || 10);
+const LIMIT = Number(process.env.LIMIT || 0); // 0 = all batches (for quick tests)
+
+const channels = [...readFileSync(CHANNELS, 'utf8').matchAll(/<channel\b[\s\S]*?<\/channel>/g)].map((m) => m[0]);
+const dir = `${TMP}/batches`;
+rmSync(dir, { recursive: true, force: true });
+mkdirSync(dir, { recursive: true });
+
+let groups = [];
+for (let i = 0; i < channels.length; i += BATCH) groups.push(channels.slice(i, i + BATCH));
+if (LIMIT) groups = groups.slice(0, LIMIT);
+console.log(`${channels.length} channels -> ${groups.length} batch(es) of ${BATCH}  (days=${DAYS}, maxConn=${MAXCONN})`);
+
+const guides = [];
+for (let i = 0; i < groups.length; i++) {
+  const cf = `${dir}/b${i}.channels.xml`;
+  const gf = `${dir}/g${i}.xml`;
+  writeFileSync(cf, `<?xml version="1.0" encoding="UTF-8"?>\n<channels>\n${groups[i].join('\n')}\n</channels>\n`);
+  await $`bun scripts/commands/epg/grab.ts --channels ${cf} --output ${gf} --days ${DAYS} --maxConnections ${MAXCONN}`
+    .cwd(EPG).quiet().nothrow();
+  const n = existsSync(gf) ? (readFileSync(gf, 'utf8').match(/<programme\b/g) || []).length : -1;
+  console.log(`  [${i + 1}/${groups.length}] ${n < 0 ? 'FAILED' : `${n} programmes`}`);
+  if (existsSync(gf)) guides.push(gf);
+}
+
+// merge: every channel is in exactly one batch, so no dedup needed
+const chans = [], progs = [];
+for (const g of guides) {
+  const t = readFileSync(g, 'utf8');
+  for (const m of t.matchAll(/<channel\b[\s\S]*?<\/channel>/g)) chans.push(m[0]);
+  for (const m of t.matchAll(/<programme\b[\s\S]*?<\/programme>/g)) progs.push(m[0]);
+}
+writeFileSync(OUT, `<?xml version="1.0" encoding="UTF-8"?>\n<tv>\n${chans.join('\n')}\n${progs.join('\n')}\n</tv>\n`);
+console.log(`merged -> ${OUT}  (${chans.length} channels, ${progs.length} programmes)`);

make-epg.mjs

@@ -0,0 +1,114 @@
+// EPG binder. Run AFTER build-iptv.mjs (refresh.ts does both + grab). It:
+//   1. matches each playlist channel to an iptv-org/epg source:
+//        - real tvg-id  -> epg channel with the same BASE id (ignoring @feed suffix)
+//        - Pluto channels -> /plu-<id>/ in the URL IS epg's pluto.tv site_id, so we
+//          synthesise a tvg-id even when upstream left it blank
+//   2. rewrites the playlist's tvg-id to a canonical id + adds the url-tvg header
+//   3. writes /tmp/iptv-org-build/custom.channels.xml for `grab`
+// Samsung & Tubi have no epg site upstream, so they only bind if a real tvg-id matches.
+import { readFileSync, writeFileSync, readdirSync, existsSync } from 'node:fs';
+import { fileURLToPath } from 'node:url';
+import path from 'node:path';
+
+const HERE = path.dirname(fileURLToPath(import.meta.url));
+const M3U = path.join(HERE, 'iptv.m3u');
+const SITES = '/tmp/iptv-org-build/epg/sites';
+const OUTXML = '/tmp/iptv-org-build/custom.channels.xml';
+// Public URL where guide.xml is served — must match deploy.ts's bucket path.
+const EPG_URL = 'https://chillidonut.com/junk/iptv/guide.xml';
+
+if (!existsSync(SITES)) {
+  process.stderr.write(`epg clone not found at ${SITES}\n` +
+    'Run: git clone --depth 1 https://github.com/iptv-org/epg.git /tmp/iptv-org-build/epg  (or use refresh.ts)\n');
+  process.exit(1);
+}
+
+const baseId = (x) => (x || '').split('@')[0].trim();
+const cleanName = (n) => n.replace(/\s*\(\d+p\)|\s*\[[^\]]*\]|\s*\([^)]*\)\s*$/g, '').trim() || n;
+
+// ---------- index epg channel mappings ----------
+const chanRe = /<channel\s+([^>]*?)>([^<]*)<\/channel>/g;
+const attrRe = /([\w]+)="([^"]*)"/g;
+const baseIdx = new Map();   // base(xmltv_id) -> {site, site_id, lang}
+const plutoIdx = new Map();  // pluto site_id (hex) -> {xmltv_id, lang}
+for (const site of readdirSync(SITES)) {
+  let dir;
+  try { dir = readdirSync(path.join(SITES, site)); } catch { continue; }
+  for (const f of dir) {
+    if (!f.endsWith('.channels.xml')) continue;
+    for (const c of readFileSync(path.join(SITES, site, f), 'utf8').matchAll(chanRe)) {
+      const a = {};
+      for (const m of c[1].matchAll(attrRe)) a[m[1]] = m[2];
+      if (site === 'pluto.tv' && a.site_id) plutoIdx.set(a.site_id, { xmltv_id: a.xmltv_id, lang: a.lang || 'en' });
+      const b = baseId(a.xmltv_id);
+      if (b && a.site_id && !baseIdx.has(b)) baseIdx.set(b, { site: a.site, site_id: a.site_id, lang: a.lang || 'en' });
+    }
+  }
+}
+
+// ---------- resolve each playlist channel ----------
+function resolve(url, tvgid, name) {
+  const pm = url.match(/plu-([0-9a-f]{16,})/i);
+  if (pm && plutoIdx.has(pm[1])) {
+    const e = plutoIdx.get(pm[1]);
+    return { id: baseId(e.xmltv_id) || `pluto-${pm[1]}`, site: 'pluto.tv', site_id: pm[1], lang: e.lang, name: cleanName(name) };
+  }
+  const b = baseId(tvgid);
+  if (b && baseIdx.has(b)) {
+    const e = baseIdx.get(b);
+    return { id: b, site: e.site, site_id: e.site_id, lang: e.lang, name: cleanName(name) };
+  }
+  return { id: b }; // keep base id if any (no grab source), else ''
+}
+
+// ---------- rewrite playlist + collect channels ----------
+const lines = readFileSync(M3U, 'utf8').split('\n');
+const channels = new Map(); // site|site_id -> row
+const seenIds = new Set();   // one grab source (and one guide <channel>) per id
+let plutoMatched = 0, idMatched = 0;
+const bySection = {};
+let section = '';
+for (let i = 0; i < lines.length; i++) {
+  const l = lines[i];
+  if (l.startsWith('# SECTION:')) section = l.replace(/.*SECTION:\s*/, '').replace(/\s{2,}.*/, '');
+  if (l === '#EXTM3U') { lines[i] = `#EXTM3U url-tvg="${EPG_URL}"`; continue; }
+  if (!l.startsWith('#EXTINF')) continue;
+  let url = '';
+  for (let j = i + 1; j < lines.length; j++) {
+    const t = lines[j].trim();
+    if (!t || t.startsWith('#')) { if (t.startsWith('#EXTINF')) break; continue; }
+    url = t; break;
+  }
+  if (!url) continue;
+  const tvgid = (l.match(/tvg-id="([^"]*)"/) || [])[1] || '';
+  const name = l.slice(l.indexOf(',') + 1);
+  const r = resolve(url, tvgid, name);
+  lines[i] = l.replace(/tvg-id="[^"]*"/, `tvg-id="${r.id || ''}"`);
+  bySection[section] = bySection[section] || { have: 0, total: 0 };
+  bySection[section].total++;
+  if (r.site && r.site_id) {
+    bySection[section].have++;
+    if (!seenIds.has(r.id)) {
+      seenIds.add(r.id);
+      if (r.site === 'pluto.tv') plutoMatched++; else idMatched++;
+      channels.set(`${r.site}|${r.site_id}`, r);
+    }
+  }
+}
+writeFileSync(M3U, lines.join('\n'));
+
+// ---------- write channels.xml ----------
+const esc = (s) => String(s).replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;').replace(/"/g, '&quot;');
+const rows = [...channels.values()].sort((a, b) => a.id.localeCompare(b.id));
+const xml = ['<?xml version="1.0" encoding="UTF-8"?>', '<channels>'];
+for (const r of rows)
+  xml.push(`  <channel site="${esc(r.site)}" lang="${esc(r.lang)}" xmltv_id="${esc(r.id)}" site_id="${esc(r.site_id)}">${esc(r.name)}</channel>`);
+xml.push('</channels>');
+writeFileSync(OUTXML, xml.join('\n') + '\n');
+
+// ---------- report ----------
+process.stderr.write(`EPG url-tvg: ${EPG_URL}\n`);
+process.stderr.write(`grab channels: ${rows.length}  (pluto: ${plutoMatched}, by-id: ${idMatched})  -> ${OUTXML}\n\n`);
+process.stderr.write('EPG-bound channels per section:\n');
+for (const [s, v] of Object.entries(bySection).sort())
+  process.stderr.write(`  ${String(v.have).padStart(4)} / ${String(v.total).padEnd(4)}  ${s}\n`);

refresh.ts

@@ -0,0 +1,33 @@
+#!/usr/bin/env bun
+// One tool (Bun), end to end: build playlist -> bind EPG ids -> grab guide.xml.
+// Uses Bun Shell ($). Clones of iptv-org/iptv & iptv-org/epg + node_modules all
+// live in /tmp (throwaway — nothing but the served files lands in the repo).
+//
+//   bun refresh.ts                # full refresh (playlist + EPG)
+//   bun refresh.ts --revalidate   # also re-probe every stream from scratch
+import { $ } from 'bun';
+import { existsSync } from 'node:fs';
+
+const HERE = import.meta.dir;
+const TMP = '/tmp/iptv-org-build';
+const EPG = `${TMP}/epg`;
+const args = process.argv.slice(2);
+
+console.log('==> 1/3 build playlist (validate streams)');
+await $`bun ${HERE}/build-iptv.mjs ${args}`;
+
+console.log('==> 2/3 bind EPG ids + write channels.xml');
+if (!existsSync(`${EPG}/.git`))
+  await $`git clone --depth 1 https://github.com/iptv-org/epg.git ${EPG}`;
+await $`bun ${HERE}/make-epg.mjs`;
+
+console.log('==> 3/3 grab guide.xml');
+if (!existsSync(`${EPG}/node_modules`)) {
+  await $`bun install`.cwd(EPG);
+  // run epg's postinstall ourselves — it fetches the iptv-org API data the
+  // grabber needs (bun install doesn't run that lifecycle hook for us).
+  await $`bun run scripts/commands/api/load.ts`.cwd(EPG);
+}
+await $`bun ${HERE}/grab-epg.ts`; // batched grab (bounded memory) -> guide.xml
+
+console.log(`==> done: ${HERE}/iptv.m3u  +  ${HERE}/guide.xml`);