# IPTV — curated playlist + EPG

Tooling that builds a personal, validated IPTV playlist from
[iptv-org/iptv](https://github.com/iptv-org/iptv) plus a matching XMLTV guide from
[iptv-org/epg](https://github.com/iptv-org/epg), then publishes the two outputs to
`chillidonut.com/junk/iptv/` (served to **TiviMate**).

This repo is **tooling only**. The build outputs (`iptv.m3u`, `guide.xml`), the
upstream clones, and `node_modules` are not committed — clones live in `/tmp`.

## Files
| File | What it does |
|------|--------------|
| `build-iptv.mjs` | Curate + validate playlist → `iptv.m3u` |
| `make-epg.mjs` | Bind `tvg-id`s to epg channels → rewrites `iptv.m3u`, writes `custom.channels.xml` |
| `grab-epg.ts` | Batched, memory-bounded EPG grab → `guide.xml` |
| `refresh.ts` | Bun orchestrator: build → bind → grab |
| `deploy.ts` | Upload `iptv.m3u` + `guide.xml` to the GCS bucket (gzipped) |

## Usage
```sh
bun refresh.ts              # build everything → iptv.m3u + guide.xml
bun refresh.ts --revalidate # also re-probe every stream from scratch
bun deploy.ts               # publish the two outputs to chillidonut.com/junk/iptv/

# individual steps:
bun build-iptv.mjs          # playlist only (--dry = counts only, no probing)
bun make-epg.mjs            # bind EPG ids (needs the epg clone in /tmp)
bun grab-epg.ts             # grab guide.xml (env: BATCH, DAYS, MAXCONN, LIMIT)
```
Requires [Bun](https://bun.com); `deploy.ts` also needs `gsutil`. Upstream clones +
the grabber's `node_modules` are created on demand under `/tmp/iptv-org-build/` —
throwaway, so a reboot (which clears `/tmp`) just means the next run re-bootstraps.

### TiviMate
Add `https://chillidonut.com/junk/iptv/iptv.m3u` as a playlist. TiviMate reads
`group-title` as categories and binds the guide via the `url-tvg` header + `tvg-id`s.

## How it works
1. **Curate** (`build-iptv.mjs`): from a clone of iptv-org/iptv `streams/*.m3u`,
   select sections — AU + AU Samsung, US broadcast + Pluto/Samsung/Tubi/Xumo,
   flagship/world News, Comedy Central, Español (Mexico + Spanish news), AR/CO/CL
   terrestrial, bespoke European terrestrial. Curated sections use name whitelists;
   name collisions get a source tag, e.g. `Comedy Central (US)` vs `(DK)`.
2. **Validate**: every stream is probed (HTTP, from this machine) and dead/geo-blocked
   ones dropped. Cached in `/tmp/iptv-org-build/validation.json` (fast re-runs;
   `--revalidate` ignores it). Probing from AU means US-geo-locked feeds fall away.
3. **Bind EPG** (`make-epg.mjs`): assigns a canonical `tvg-id` per channel — by base
   id (ignoring iptv-org's `@feed` suffix), and for **Pluto** by the `plu-<id>` in the
   URL, which *is* epg's `pluto.tv` `site_id` (so blank-`tvg-id` Pluto channels still
   get a guide). Writes `custom.channels.xml` (one source per id).
4. **Grab** (`grab-epg.ts`): runs iptv-org/epg's grabber over `custom.channels.xml`,
   batched, → `guide.xml`. See the memory note below.

### EPG coverage / freshness
~1,000 of ~2,000 channels are guide-bound. The rest have no usable id: many
Samsung/Tubi/Xumo FAST channels ship a blank `tvg-id`, and iptv-org/epg has **no
Samsung or Tubi site**. XMLTV does **not** loop — programmes have absolute times, so
the guide goes stale (FAST ~12–48 h, broadcasters ~3–7 days) and must be regenerated
on a schedule (daily). A daily `bun refresh.ts && bun deploy.ts` job is the intended
setup (not yet wired up — see memory note).

## Deploying
`bun deploy.ts` uploads only `iptv.m3u` + `guide.xml` to `gs://chillidonut.com/junk/iptv/`:
- **`cp`, not `rsync`** — it adds/overwrites just these two objects; everything else
  in the bucket's `junk/` is left untouched (never synced or deleted).
- **`-Z`** gzips on upload and sets `Content-Encoding: gzip` (~32 MB guide → ~3–4 MB;
  TiviMate decompresses transparently). `url-tvg` stays `…/guide.xml` — don't pre-gzip,
  that would double-compress.

(On chillidonut.com, `static/junk/**` is excluded from `hugo deploy`, which is why
these go straight to the bucket instead of through the site build.)

## Known limitations / notes for later

### ⚠️ The EPG grab is very memory-hungry
The iptv-org/epg grabber loads the **entire** iptv-org API dataset into memory and
retains all fetched programme data until it serializes at the end. For ~1,000
channels a single process climbs to **~20 GB+**.

`grab-epg.ts` works around this by splitting into batches that each run as a
**separate `bun` process** (memory reclaimed when each exits). That bounds the
*sustained* footprint but **not** the per-batch peak: the baseline API load alone is
several GB, and some sites retain a lot, so individual batches have spiked to **~18 GB**
even at `BATCH=25`. Programme volume is tiny (≤~2k/batch), so this is grabber overhead,
not our data — shrinking `BATCH` barely helps.

**Not yet suitable for an unattended daily job on a working machine.** Ideas to make it
resource-efficient (none implemented — batching is a stopgap):
- **Trim the API load**: pre-filter so the grabber loads only our channels' data, not
  the whole iptv-org dataset (kills the multi-GB baseline).
- **Stream to disk**: fork the grab to write each channel's programmes incrementally
  and release them, instead of one giant in-memory guide.
- **Run it off-machine**: do the grab in CI/cloud (iptv-org/epg itself runs as a daily
  GitHub Action) and just publish the resulting `guide.xml`.
- **Don't self-grab**: point `url-tvg` at a maintained hosted XMLTV for channels that
  have one, and only self-grab the gaps.

### Other
- Samsung/Tubi (and most blank-`tvg-id` FAST channels) have no EPG — see above.
- Validation reflects *this machine's* connectivity (currently AU). Re-run with
  `--revalidate` from a VPN to re-evaluate US-geo feeds.