progress

ARCHITECHTURE.md

@@ -1,21 +1,23 @@
-devbox for shareable development environment tooling
+nodejs for everything
+
+Node version manager (nvm) to install node
 
 git monorepo for housing separate packages within a single repository (see ./services and ./packages)
 
 pnpm for package management and workspaces (separate node packages.)
 
-Phoenix framework
-
 docker-compose for containerized development
 
-Kamal for deployments
+Komodo for deployments
 
-ggshield for preventing git commits containing secrets
+ggshield for preventing accidental git commits containing secrets
 
 direnv for loading .envrc
 
 Graphile Worker for work queue, cron
 
+nano-spawn or execa to run any non-node programs like yolo or 
+
 Postgres for data storage
 
 S3 for media storage

README.md

@@ -2,47 +2,30 @@
 
 ![Tests Status](https://gitea.futureporn.net/futureporn/fp/actions/workflows/tests.yaml/badge.svg)
 ![Build Status](https://gitea.futureporn.net/futureporn/fp/actions/workflows/builder.yaml/badge.svg)
-![Elixir](https://img.shields.io/badge/elixir-%234B275F.svg?style=for-the-badge&logo=elixir&logoColor=white)
 ![Postgres](https://img.shields.io/badge/postgres-%23316192.svg?style=for-the-badge&logo=postgresql&logoColor=white)
-[![Built with Devbox](https://www.jetify.com/img/devbox/shield_galaxy.svg)](https://www.jetify.com/devbox/docs/contributor-quickstart/)
 
 Source Code for https://futureporn.net
 
 See ./ARCHITECTURE.md for an overview of the infrastructure components. 
 
-## Getting Started
+## Dev notes
 
-The main gist is as follows.
+### backup/restore dev database
-
-1. install [docker](https://docs.docker.com/engine/install) `wget -O- get.docker.com | bash`
-1. Install [devbox](https://www.jetify.com/devbox/docs/installing_devbox/) `curl -fsSL https://get.jetify.com/devbox | bash`
-2. Install development environment & packages using devbox.
-
-    devbox install
-
-
-3. Run database and other accessories with `docker compose up --watch` 
-4. In another terminal, run the phoenix "bright" app with `devbox run bright:dev`
-4. Visit http://localhost:4000
-
-If all went well, editing source code will automatically affect the website running in your browser.
-
-## backup/restore dev database
 
 @see https://stackoverflow.com/a/29913462/1004931
 
-### backup
+#### backup
 
 Use devbox helper script
 
     devbox run backup
 
-### restore
+#### restore
 
     cat ./backups/your-backup.sql | docker exec -i postgres_db psql -U postgres
 
 
-## testing
+### testing
 
 there is some undesirable behavior when running tests because nektos/act mimicks github actions.
 we are banned from github so we aren't using that. instead, we use gitea act_runner.

services/our/Dockerfile

@@ -26,12 +26,15 @@ RUN wget -q https://github.com/shaka-project/shaka-packager/releases/download/v3
 COPY --from=ipfs/kubo:v0.36.0 /usr/local/bin/ipfs /usr/local/bin/ipfs
 RUN ipfs init
 
-
 # Bundle the vibeui pytorch model
 RUN mkdir -p /app/vibeui \
   && wget -q https://gitea.futureporn.net/futureporn/fp/raw/branch/main/apps/vibeui/public/vibeui.pt -O /app/vibeui/vibeui.pt \
   && wget -q https://gitea.futureporn.net/futureporn/fp/raw/branch/main/apps/vibeui/public/data.yaml -O /app/vibeui/data.yaml
 
+# Install openwhisper
+COPY --from=ghcr.io/ggml-org/whisper.cpp:main-e7bf0294ec9099b5fc21f5ba969805dfb2108cea /app /app/whisper.cpp
+ENV PATH="$PATH:/app/whisper.cpp/build/bin"
+
 # Copy and install dependencies
 COPY package.json package-lock.json ./
 RUN npm install --ignore-scripts=false --foreground-scripts --verbose

services/our/README.md

@@ -1,5 +1,34 @@
 # futureporn
 
+https://future.porn
+
+
+## Software Dependencies
+
+* node
+* pnpm
+* ffmpeg
+* [whisper-cli](https://github.com/ggml-org/whisper.cpp)
+* [shaka-packager](https://github.com/shaka-project/shaka-packager/releases)
+
+
+## Getting started (developers only)
+
+Ensure you have all the above software dependencies available on system PATH
+
+Install node packages
+
+    pnpm install
+
+Start docker containers 
+
+    docker compose -f ./compose.development.yaml up
+
+Start node app in dev mode. Env vars must be available to the app-- We're using dotenvx to load them.
+
+    dotenvx run -f ../../.env.development.local -- pnpm run dev
+
+
 ## projekt requirements
 
 * [x] NO BUNDLER (esbuild/vite/webpack/parcel/swc/etc.). IF YOU REACH FOR A BUNDLER, YOU'RE OVERCOMPLICATING IT!
@@ -21,8 +50,8 @@
 ## Tiers & Privs
 
 * user - view, torrent, download
-* supporterTier1 - view, torrent, download, adfree, upload
+* supporterTier1 - view, torrent, download, adfree, upload, vibeui, closed captions, search
-* supporterTier6 - view, torrent, download, adfree, upload, csv, sql
+* supporterTier6 - view, torrent, download, adfree, upload, vibeui, closed captions, search, csv, sql, pytorch
 
 ## troubleshooting
 
@@ -84,7 +113,7 @@ sharp is often a pain in the ass to install.
 ```
 
 
-If you have trouble installing sharp, try ignoring the system's installed libvips.
+If you have trouble installing sharp, try ignoring the system's installed libvips. This usually needs to be done after every time npm installs a new package.
 
     SHARP_IGNORE_GLOBAL_LIBVIPS=1 npm install --ignore-scripts=false --foreground-scripts --verbose --platform=linux --arch=x64 sharp
 
@@ -92,7 +121,6 @@ If you have trouble installing sharp, try ignoring the system's installed libvip
 
 Actually, better advice is to probably **remove libvips from the system**. This way, a compatible libvips is always pulled during `npm install`.
 
-Annoyingly, it might be necessary to re-install sharp after every new npm package, even if said package is unrelated to sharp.
 
 #### edgesOut??
 
@@ -153,13 +181,18 @@ npm verbose code 1
 npm error A complete log of this run can be found in: /home/cj/.npm/_logs/2025-07-14T12_49_09_670Z-debug-0.log
 ```
 
+## Development
+
+### Apply migrations
+
+    dotenvx run -f ../../.env.development.local -- npx prisma migrate dev --name "rename_asrvtt"
 
 ## Deployments
 
 ### Apply migrations
 
     cd /opt/futureporn/services/our
-    npx @dotenvx/dotenvx run -f /usr/local/etc/futureporn/our/env -- npx prisma migrate deploy
+    npx @dotenvx/dotenvx run -f /usr/local/etc/futureporn/our/.env -- npx prisma migrate deploy
 
 ### pgweb
 

services/our/compose.development.yaml

@@ -5,7 +5,7 @@ services:
     container_name: our-postgres
     image: postgres:17
     restart: unless-stopped
-    env_file: ./../../.env.development
+    env_file: ./../../.env.development.local
     ports:
       - "5432:5432"
     volumes:

services/our/package.json

@@ -4,14 +4,15 @@
 	"version": "2.0.1",
 	"type": "module",
 	"scripts": {
-		"dev": "concurrently npm:dev:serve npm:dev:build npm:dev:worker",
+		"dev": "concurrently npm:dev:serve npm:dev:build npm:dev:worker npm:dev:compose",
-		"dev:serve": "tsx watch ./src/index.ts",
+		"dev:serve": "npx @dotenvx/dotenvx run -f ../../.env.development.local -- tsx watch ./src/index.ts",
+		"dev:compose": "docker compose -f compose.development.yaml up",
+		"dev:worker": "npx @dotenvx/dotenvx run -e GRAPHILE_LOGGER_DEBUG=1 -f ../../.env.development.local -- tsx watch ./src/worker.ts",
+		"dev:build": "chokidar 'src/**/*.{js,ts}' -c tsup --clean",
 		"start": "echo please use either start:server or start:worker; exit 1",
 		"start:server": "tsx ./src/index.ts",
 		"start:worker": "tsx ./src/worker.ts",
 		"preview": "vite preview",
-		"dev:worker": "GRAPHILE_LOGGER_DEBUG=1 tsx watch ./src/worker.ts",
-		"dev:build": "chokidar 'src/**/*.{js,ts}' -c tsup --clean",
 		"build": "tsup --clean",
 		"lint": "eslint .",
 		"clean": "rm -rf node_modules && rm -rf pnpm-lock.yaml",
@@ -78,11 +79,15 @@
 		"nanoid": "^5.1.5",
 		"node-fetch": "^3.3.2",
 		"onnxruntime-node": "1.22.0-rev",
+		"pino": "^9.7.0",
+		"pino-pretty": "^13.0.0",
+		"pnpm": "^10.14.0",
 		"protobufjs": "^7.5.3",
 		"rate-limiter-flexible": "^7.1.1",
 		"rimraf": "6.0.1",
 		"sharp": "^0.34.3",
 		"slugify": "^1.6.6",
+		"slvtt": "^0.3.4",
 		"ts-node": "^10.9.2",
 		"tsup": "^8.5.0",
 		"tsx": "^4.20.3",
@@ -92,4 +97,4 @@
 	"prisma": {
 		"seed": "tsx prisma/seed.ts"
 	}
-}
+}

services/our/pnpm-lock.yaml

@@ -131,6 +131,12 @@ importers:
       onnxruntime-node:
         specifier: 1.22.0-rev
         version: 1.22.0-rev
+      pino:
+        specifier: ^9.7.0
+        version: 9.7.0
+      pino-pretty:
+        specifier: ^13.0.0
+        version: 13.0.0
       protobufjs:
         specifier: ^7.5.3
         version: 7.5.3
@@ -1743,6 +1749,9 @@ packages:
     resolution: {integrity: sha512-1rXeuUUiGGrykh+CeBdu5Ie7OJwinCgQY0bc7GCRxy5xVHy+moaqkpL/jqQq0MtQOeYcrqEz4abc5f0KtU7W4A==}
     engines: {node: '>=12.5.0'}
 
+  colorette@2.0.20:
+    resolution: {integrity: sha512-IfEDxwoWIjkeXL1eXcDiow4UbKjhLdq6/EuSVR9GMN7KVH3r9gQ83e73hsz1Nd1T3ijd5xv1wcWRYO+D6kCI2w==}
+
   combined-stream@1.0.8:
     resolution: {integrity: sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==}
     engines: {node: '>= 0.8'}
@@ -1806,6 +1815,9 @@ packages:
   date-fns@4.1.0:
     resolution: {integrity: sha512-Ukq0owbQXxa/U3EGtsdVBkR1w7KOQ5gIBqdH2hkvknzZPYvBxb/aa6E8L7tmjFtkwZBu3UXBbjIgPo/Ez4xaNg==}
 
+  dateformat@4.6.3:
+    resolution: {integrity: sha512-2P0p0pFGzHS5EMnhdxQi7aJN+iMheud0UhG4dlE1DLAlvL8JHjJJTX/CSm4JXwV0Ka5nGk3zC5mcb5bUQUxxMA==}
+
   debug@4.4.1:
     resolution: {integrity: sha512-KcKCqiftBJcZr++7ykoDIEwSa3XWowTfNPo92BYxjXiyYEVrUQh2aLyhxBCwww+heortUFxEJYcRzosstTEBYQ==}
     engines: {node: '>=6.0'}
@@ -2010,6 +2022,9 @@ packages:
     resolution: {integrity: sha512-i6FbQ0ZUPV6yhFSRI2SQBEqJzoWDiN4cnulTT2jm0f0lUIXg8/iPebACCrOY80rggd9LaSU65GFOI/xnJBdzyA==}
     engines: {node: '>=16'}
 
+  fast-copy@3.0.2:
+    resolution: {integrity: sha512-dl0O9Vhju8IrcLndv2eU4ldt1ftXMqqfgN4H1cpmGV7P6jeB9FwpN9a2c8DPGE1Ys88rNUJVYDHq73CGAGOPfQ==}
+
   fast-decode-uri-component@1.0.1:
     resolution: {integrity: sha512-WKgKWg5eUxvRZGwW8FvfbaH7AXSh2cL+3j5fMGzUMCxWBJ3dV3a7Wz8y2f/uQ0e3B6WmodD3oS54jTQ9HVTIIg==}
 
@@ -2039,6 +2054,9 @@ packages:
     resolution: {integrity: sha512-dwsoQlS7h9hMeYUq1W++23NDcBLV4KqONnITDV9DjfS3q1SgDGVrBdvvTLUotWtPSD7asWDV9/CmsZPy8Hf70A==}
     engines: {node: '>=6'}
 
+  fast-safe-stringify@2.1.1:
+    resolution: {integrity: sha512-W+KJc2dmILlPplD/H4K9l9LcAHAfPtP6BY84uVLXQ6Evcz9Lcg33Y2z1IVblT6xdY54PXYVHEv+0Wpq8Io6zkA==}
+
   fast-uri@3.0.6:
     resolution: {integrity: sha512-Atfo14OibSv5wAp4VWNsFYE1AchQRTv9cBGWET4pZWHzYshFSS9NQI6I57rdKn9croWVMbYFbLhJ+yJvmZIIHw==}
 
@@ -2241,6 +2259,9 @@ packages:
     resolution: {integrity: sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==}
     engines: {node: '>= 0.4'}
 
+  help-me@5.0.0:
+    resolution: {integrity: sha512-7xgomUX6ADmcYzFik0HzAxh/73YlKR9bmFzf51CZwR+b6YtzU2m0u49hQCqV6SvlqIqsaxovfwdvbnsw3b/zpg==}
+
   http-errors@2.0.0:
     resolution: {integrity: sha512-FtwrG/euBzaEjYeRqOgly7G0qviiXoJWnvEH2Z1plBdXgbyjv34pHTSb9zoeHMyDy33+DWy5Wt9Wo+TURtOYSQ==}
     engines: {node: '>= 0.8'}
@@ -2809,6 +2830,10 @@ packages:
   pino-abstract-transport@2.0.0:
     resolution: {integrity: sha512-F63x5tizV6WCh4R6RHyi2Ml+M70DNRXt/+HANowMflpgGFMAym/VKm6G7ZOQRjqN7XbGxK1Lg9t6ZrtzOaivMw==}
 
+  pino-pretty@13.0.0:
+    resolution: {integrity: sha512-cQBBIVG3YajgoUjo1FdKVRX6t9XPxwB9lcNJVD5GCnNM4Y6T12YYx8c6zEejxQsU0wrg9TwmDulcE9LR7qcJqA==}
+    hasBin: true
+
   pino-std-serializers@7.0.0:
     resolution: {integrity: sha512-e906FRY0+tV27iq4juKzSYPbUj2do2X2JX4EzSca1631EB2QJQUqGbDuERal7LCtOpxl6x3+nvo9NPZcmjkiFA==}
 
@@ -3024,6 +3049,9 @@ packages:
     resolution: {integrity: sha512-b3rppTKm9T+PsVCBEOUR46GWI7fdOs00VKZ1+9c1EWDaDMvjQc6tUwuFyIprgGgTcWoVHSKrU8H31ZHA2e0RHA==}
     engines: {node: '>=10'}
 
+  secure-json-parse@2.7.0:
+    resolution: {integrity: sha512-6aU+Rwsezw7VR8/nyvKTx8QpWH9FrcYiXXlqC4z5d5XQBDRqtbfsRjnwGyqbi3gddNtWHuEk9OANUotL26qKUw==}
+
   secure-json-parse@3.0.2:
     resolution: {integrity: sha512-H6nS2o8bWfpFEV6U38sOSjS7bTbdgbCGU9wEM6W14P5H0QOsz94KCusifV44GpHDTu2nqZbuDNhTzu+mjDSw1w==}
 
@@ -5579,6 +5607,8 @@ snapshots:
       color-convert: 2.0.1
       color-string: 1.9.1
 
+  colorette@2.0.20: {}
+
   combined-stream@1.0.8:
     dependencies:
       delayed-stream: 1.0.0
@@ -5632,6 +5662,8 @@ snapshots:
 
   date-fns@4.1.0: {}
 
+  dateformat@4.6.3: {}
+
   debug@4.4.1(supports-color@5.5.0):
     dependencies:
       ms: 2.1.3
@@ -5879,6 +5911,8 @@ snapshots:
       node-addon-api: 8.5.0
       prebuild-install: 7.1.3
 
+  fast-copy@3.0.2: {}
+
   fast-decode-uri-component@1.0.1: {}
 
   fast-deep-equal@3.1.3: {}
@@ -5912,6 +5946,8 @@ snapshots:
 
   fast-redact@3.5.0: {}
 
+  fast-safe-stringify@2.1.1: {}
+
   fast-uri@3.0.6: {}
 
   fast-xml-parser@4.4.1:
@@ -6167,6 +6203,8 @@ snapshots:
     dependencies:
       function-bind: 1.1.2
 
+  help-me@5.0.0: {}
+
   http-errors@2.0.0:
     dependencies:
       depd: 2.0.0
@@ -6670,6 +6708,22 @@ snapshots:
     dependencies:
       split2: 4.2.0
 
+  pino-pretty@13.0.0:
+    dependencies:
+      colorette: 2.0.20
+      dateformat: 4.6.3
+      fast-copy: 3.0.2
+      fast-safe-stringify: 2.1.1
+      help-me: 5.0.0
+      joycon: 3.1.1
+      minimist: 1.2.8
+      on-exit-leak-free: 2.1.2
+      pino-abstract-transport: 2.0.0
+      pump: 3.0.3
+      secure-json-parse: 2.7.0
+      sonic-boom: 4.2.0
+      strip-json-comments: 3.1.1
+
   pino-std-serializers@7.0.0: {}
 
   pino@9.7.0:
@@ -6898,6 +6952,8 @@ snapshots:
 
   safe-stable-stringify@2.5.0: {}
 
+  secure-json-parse@2.7.0: {}
+
   secure-json-parse@3.0.2: {}
 
   secure-json-parse@4.0.0: {}

services/our/prisma/migrations/20250729003839_rename_asrvtt/migration.sql

@@ -0,0 +1,9 @@
+/*
+  Warnings:
+
+  - You are about to drop the column `asrVtt` on the `Vod` table. All the data in the column will be lost.
+
+*/
+-- AlterTable
+ALTER TABLE "Vod" DROP COLUMN "asrVtt",
+ADD COLUMN     "asrVttKey" TEXT;

services/our/prisma/migrations/20250803040941_add_slvtt/migration.sql

@@ -0,0 +1,3 @@
+-- AlterTable
+ALTER TABLE "Vod" ADD COLUMN     "slvttSheetKeys" JSONB,
+ADD COLUMN     "vttKey" TEXT;

services/our/prisma/migrations/20250803041206_use_slvtt_prefix/migration.sql

@@ -0,0 +1,9 @@
+/*
+  Warnings:
+
+  - You are about to drop the column `vttKey` on the `Vod` table. All the data in the column will be lost.
+
+*/
+-- AlterTable
+ALTER TABLE "Vod" DROP COLUMN "vttKey",
+ADD COLUMN     "slvttVTTKey" TEXT;

services/our/prisma/schema.prisma

@@ -71,17 +71,20 @@ model Vod {
   uploaderId String // previously in Upload
   uploader   User    @relation(fields: [uploaderId], references: [id])
 
-  streamDate  DateTime
+  streamDate     DateTime
-  notes       String?
+  notes          String?
-  segmentKeys Json?
+  segmentKeys    Json?
-  sourceVideo String?
+  sourceVideo    String?
-  hlsPlaylist String?
+  hlsPlaylist    String?
-  thumbnail   String?
+  thumbnail      String?
-  asrVtt      String?
+  asrVttKey      String?
-  status      VodStatus @default(pending)
+  slvttSheetKeys Json?
-  sha256sum   String?
+  slvttVTTKey    String?
-  cidv1       String?
+
-  funscript   String?
+  status    VodStatus @default(pending)
+  sha256sum String?
+  cidv1     String?
+  funscript String?
 
   createdAt DateTime @default(now())
   updatedAt DateTime @updatedAt

services/our/src/app.ts

@@ -72,6 +72,7 @@ export function buildApp() {
         return new Handlebars.SafeString(text);
     });
     Handlebars.registerHelper('getCdnUrl', function (s3Key) {
+        // Before you remove this console.log, find a way to memoize this function!
         console.log(`getCdnUrl called with CDN_ORIGIN=${env.CDN_ORIGIN} and CDN_TOKEN_SECRET=${env.CDN_TOKEN_SECRET}`)
         return signUrl(`${env.CDN_ORIGIN}/${s3Key}`, {
             securityKey: env.CDN_TOKEN_SECRET,

services/our/src/config/env.ts

@@ -1,8 +1,8 @@
 // ./config/env.ts
 import { z } from 'zod';
-import dotenvx from '@dotenvx/dotenvx'
+// import dotenvx from '@dotenvx/dotenvx'
 
-dotenvx.config({ path: ['../../.env.development'] })
+// dotenvx.config({ path: ['../../.env.development'] })
 // if (process.env.NODE_ENV === 'development') {
 // }
 
@@ -28,7 +28,9 @@ const EnvSchema = z.object({
     CDN_TOKEN_SECRET: z.string(),
     CACHE_ROOT: z.string().default('/tmp/our'),
     VIBEUI_DIR: z.string().default('/opt/futureporn/apps/vibeui'),
-    APP_DIR: z.string().default('/app')
+    APP_DIR: z.string().default('/app'),
+    WHISPER_DIR: z.string(),
+    LOG_LEVEL: z.string().default('info'),
 });
 
 const parsed = EnvSchema.safeParse(process.env);

services/our/src/index.ts

@@ -1,7 +1,8 @@
 import { env } from './config/env'
 import { buildApp } from "./app.js";
+import logger from './utils/logger.js'
 
 const app = buildApp()
 
-console.log(`app listening on port ${env.PORT}`)
+logger.info(`app listening on port ${env.PORT}`)
-app.listen({ port: env.PORT, host: '0.0.0.0' })
+app.listen({ port: env.PORT, host: '0.0.0.0' })

services/our/src/plugins/index.ts

@@ -3,6 +3,7 @@ import { withAccelerate } from "@prisma/extension-accelerate"
 import { type FastifyInstance, type FastifyReply, type FastifyRequest } from 'fastify'
 import { env } from '../config/env'
 import { constants } from '../config/constants'
+import { getTargetUser } from '../utils/authorization'
 
 const prisma = new PrismaClient().$extends(withAccelerate())
 
@@ -102,4 +103,17 @@ export default async function indexRoutes(fastify: FastifyInstance): Promise<voi
     fastify.get('/version', function (request, reply) {
         return reply.send(constants.site.version)
     })
+    fastify.get('/pricing', async function (request, reply) {
+        const user = await getTargetUser(request, reply)
+        const cdnOrigin = env.CDN_ORIGIN;
+        const NODE_ENV = env.NODE_ENV;
+        const authPath = env.PATREON_AUTHORIZE_PATH
+        return reply.view('pricing.hbs', {
+            user,
+            cdnOrigin,
+            NODE_ENV,
+            authPath,
+            site: constants.site,
+        })
+    })
 }

services/our/src/tasks/createStoryboard.ts

@@ -0,0 +1,114 @@
+
+import type { Task, Helpers } from "graphile-worker";
+import { PrismaClient } from "../../generated/prisma";
+import { withAccelerate } from "@prisma/extension-accelerate";
+import { getOrDownloadAsset } from "../utils/cache";
+import { env } from "../config/env";
+import { S3Client } from "@aws-sdk/client-s3";
+import { getS3Client, uploadFile } from "../utils/s3";
+import { nanoid } from "nanoid";
+import { getNanoSpawn } from "../utils/nanoSpawn";
+import { basename, join } from "node:path";
+import { mkdir } from "fs-extra";
+import { generateClosedCaptions } from "../utils/transcription";
+import logger from "../utils/logger";
+import { create, type SLVTTOptions } from 'slvtt';
+import { readdir } from "node:fs/promises";
+import { concurrency } from "sharp";
+
+const prisma = new PrismaClient().$extends(withAccelerate());
+
+
+interface Payload {
+  vodId: string;
+}
+
+
+function assertPayload(payload: any): asserts payload is Payload {
+  if (typeof payload !== "object" || !payload) throw new Error("invalid payload-- was not an object.");
+  if (typeof payload.vodId !== "string") throw new Error("invalid payload-- was missing vodId");
+}
+
+
+export default async function createStoryboard(payload: any, helpers: Helpers) {
+  assertPayload(payload)
+  const { vodId } = payload
+  const vod = await prisma.vod.findFirstOrThrow({
+    where: {
+      id: vodId
+    },
+    select: {
+      sourceVideo: true,
+      slvttSheetKeys: true,
+      slvttVTTKey: true,
+    },
+  })
+  const taskId = nanoid()
+
+  if (vod.slvttVTTKey) {
+    logger.info(`Doing nothing-- vod ${vodId} already has a slvttVTTKey.`)
+    return; // Exit the function early
+  }
+
+  if (!vod.sourceVideo) {
+    throw new Error(`Failed to create vtt-- vod ${vodId} is missing a sourceVideo.`);
+  }
+
+
+
+
+  const s3Client = getS3Client()
+
+  logger.debug(`downloading ${vod.sourceVideo}. CACHE_ROOT=${env.CACHE_ROOT}`)
+  const videoFilePath = await getOrDownloadAsset(s3Client, env.S3_BUCKET, vod.sourceVideo)
+
+  logger.debug(`running slvtt on VOD vodId=${vodId}, videoFilePath=${videoFilePath}`)
+
+  const outputDirectory = join(env.CACHE_ROOT, nanoid())
+
+  const options: SLVTTOptions = {
+    videoFilePath,
+    outputDirectory,
+    cols: 9,
+    rows: 6,
+    frameHeight: 78,
+    frameWidth: 140,
+    concurrencyLimit: 15,
+    numSamples: 696,
+  }
+
+  await create(options);
+
+
+  logger.debug(`Uploading slvtt videoFrameSheets for vodId=${vodId}`)
+  const files = await readdir(outputDirectory)
+  const sheets = files.filter((f) => f.endsWith('.webp'));
+  const vtts = files.filter((f) => f.endsWith('.vtt'))
+  if (vtts.length === 0) {
+    throw new Error('No .vtt found in the slvtt output. This should never happen.');
+  }
+  logger.debug(`slvtt created the following files.`)
+  logger.debug(files)
+
+  let slvttSheetKeys: string[] = [];
+  for (const sheet of sheets) {
+    logger.debug(`Uploading sheet=${JSON.stringify(sheet)}`);
+    let sheetKey = await uploadFile(s3Client, env.S3_BUCKET, `slvtt/${taskId}/${sheet}`, join(outputDirectory, sheet), 'image/webp')
+    slvttSheetKeys.push(sheetKey);
+  }
+
+  logger.debug(`Uploading slvtt vtt for vodId=${vodId}`)
+  const vtt = vtts[0]
+  const slvttVTTKey = await uploadFile(s3Client, env.S3_BUCKET, `slvtt/${taskId}/${vtt}`, join(outputDirectory, vtt), 'text/vtt')
+
+  logger.debug(`updating vod ${vodId} record. slvttSheetKeys=${JSON.stringify(slvttSheetKeys)}, slvttVTTKey=${slvttVTTKey}`);
+
+  await prisma.vod.update({
+    where: { id: vodId },
+    data: {
+      slvttSheetKeys,
+      slvttVTTKey,
+    }
+  });
+
+}

services/our/src/tasks/createTranscription.ts

@@ -1,43 +1,86 @@
+/** ideas
+ * - https://github.com/ggml-org/whisper.cpp/tree/master/examples/cli
+ * - https://whisperapi.com
+ * - https://elevenlabs.io/pricing
+ * - https://easy-peasy.ai/#pricing
+ * - https://www.clipto.com/pricing
+ * - https://github.com/m-bain/whisperX
+ * - https://github.com/kaldi-asr/kaldi
+ * - https://github.com/usefulsensors/moonshine
+ * - https://docs.bunny.net/reference/video_transcribevideo
+ */
 
 
-// const transcribe = async (helpers: Helpers) => {
+import type { Task, Helpers } from "graphile-worker";
-//   /** ideas
+import { PrismaClient } from "../../generated/prisma";
-//    * - https://whisperapi.com
+import { withAccelerate } from "@prisma/extension-accelerate";
-//    * - https://elevenlabs.io/pricing
+import { getOrDownloadAsset } from "../utils/cache";
-//    * - https://easy-peasy.ai/#pricing
+import { env } from "../config/env";
-//    * - https://www.clipto.com/pricing
+import { S3Client } from "@aws-sdk/client-s3";
-//    * - https://github.com/m-bain/whisperX
+import { getS3Client, uploadFile } from "../utils/s3";
-//    * - https://github.com/kaldi-asr/kaldi
+import { nanoid } from "nanoid";
-//    * - https://github.com/usefulsensors/moonshine
+import { getNanoSpawn } from "../utils/nanoSpawn";
-//    * - https://docs.bunny.net/reference/video_transcribevideo
+import { basename, join } from "node:path";
-//    */
+import { mkdir } from "fs-extra";
-//   helpers.logger.warn('@todo transcribe')
+import { generateClosedCaptions } from "../utils/transcription";
-// }
+import logger from "../utils/logger";
 
-// const createHlsPlaylist = async (helpers: Helpers) => {
+const prisma = new PrismaClient().$extends(withAccelerate());
-//   helpers.logger.warn('@todo createHlsPlaylist')
-// }
 
 
-// /**
-//  * Identify the tasks we need to do
-//  */
-// function identify() {
-
-// }
-import { Helpers } from "graphile-worker";
-
 interface Payload {
   vodId: string;
 }
 
+
 function assertPayload(payload: any): asserts payload is Payload {
   if (typeof payload !== "object" || !payload) throw new Error("invalid payload-- was not an object.");
   if (typeof payload.vodId !== "string") throw new Error("invalid payload-- was missing vodId");
 }
 
 
-export default async function createTranscription(payload: any, helpers: Helpers) {
+export default async function transcribeVideo(payload: any, helpers: Helpers) {
   assertPayload(payload)
-  helpers.logger.warn(`@TODO createTranscription`)
+  const { vodId } = payload
+  const vod = await prisma.vod.findFirstOrThrow({
+    where: {
+      id: vodId
+    },
+    select: {
+      sourceVideo: true,
+      asrVttKey: true,
+    },
+  })
+
+  if (vod.asrVttKey) {
+    logger.info(`Doing nothing-- vod ${vodId} already has a vtt.`)
+    return; // Exit the function early
+  }
+
+  if (!vod.sourceVideo) {
+    throw new Error(`Failed to create vtt-- vod ${vodId} is missing a sourceVideo.`);
+  }
+
+
+
+
+  const s3Client = getS3Client()
+
+  logger.debug(`download video segments from pull-thru cache`)
+  const videoFilePath = await getOrDownloadAsset(s3Client, env.S3_BUCKET, vod.sourceVideo)
+
+  logger.debug(`Transcribing VOD vodId=${vodId}, videoFilePath=${videoFilePath}`)
+  const captionsFilePath = await generateClosedCaptions(videoFilePath)
+  const captionsFileBasename = basename(captionsFilePath)
+
+  logger.debug(`Uploading closed captions for vodId=${vodId}`)
+  const asrVttKey = await uploadFile(s3Client, env.S3_BUCKET, captionsFileBasename, captionsFilePath, 'text/vtt')
+
+  logger.debug(`updating vod ${vodId} record. asrVttKey=${asrVttKey}`);
+
+  await prisma.vod.update({
+    where: { id: vodId },
+    data: { asrVttKey }
+  });
+
 }

services/our/src/tasks/createVideoThumbnail.ts

@@ -48,11 +48,6 @@ async function createThumbnail(helpers: Helpers, inputFilePath: string) {
     cwd: env.APP_DIR,
   });
 
-  // const exitCode = await subprocess;
-  // if (exitCode !== 0) {
-  //   console.error(`vcsi failed with exit code ${exitCode}`);
-  //   process.exit(exitCode);
-  // }
   helpers.logger.debug('result as follows')
   helpers.logger.debug(JSON.stringify(result, null, 2))
 

services/our/src/tasks/scheduleVodProcessing.ts

@@ -43,9 +43,10 @@ const scheduleVodProcessing: Task = async (payload: unknown, helpers) => {
   if (!vod.sha256sum) jobs.push(helpers.addJob("generateVideoChecksum", { vodId }));
   if (!vod.thumbnail) jobs.push(helpers.addJob("createVideoThumbnail", { vodId }));
   if (!vod.hlsPlaylist) jobs.push(helpers.addJob("createHlsPlaylist", { vodId }));
-  if (!vod.asrVtt) jobs.push(helpers.addJob("createAsrVtt", { vodId }));
   if (!vod.cidv1) jobs.push(helpers.addJob("createIpfsCid", { vodId }));
   if (!vod.funscript) jobs.push(helpers.addJob("createFunscript", { vodId }));
+  if (!vod.asrVttKey) jobs.push(helpers.addJob("createTranscription", { vodId }));
+  if (!vod.slvttVTTKey) jobs.push(helpers.addJob("createStoryboard", { vodId }));
 
   const changes = jobs.length;
   if (changes > 0) {

services/our/src/tests/spriteVideo.integration.test.ts

@@ -0,0 +1,51 @@
+import { describe, it, expect, beforeEach, afterEach } from "vitest";
+import { generateStoryboards } from "../utils/spriteVideo";
+import fs from "fs";
+import path from "path";
+import { env } from "../config/env";
+import logger from "../utils/logger";
+import { nanoid } from "nanoid";
+
+const __dirname = import.meta.dirname;
+logger.info(`NODE_ENV=${env.NODE_ENV}`);
+
+const TEST_VIDEO = path.resolve(__dirname, "fixtures/sample-projektmelody-chaturbate-2025-07-21.mp4");
+// const TEST_VIDEO = path.resolve(`/home/cj/Documents/futureporn-meta/recordings/projektmelody-chaturbate-2025-07-21.mp4`)
+const BASE_TEMP_DIR = path.join(env.CACHE_ROOT, "sprites_test_output");
+
+function getTempPrefix(testName: string): string {
+  return path.join(BASE_TEMP_DIR, testName);
+}
+
+function cleanDir(dir: string) {
+  if (fs.existsSync(dir)) {
+    fs.rmSync(dir, { recursive: true, force: true });
+  }
+}
+
+describe("generateSprites", () => {
+  beforeEach(() => {
+    if (!fs.existsSync(TEST_VIDEO)) {
+      throw new Error(`Test video not found at ${TEST_VIDEO}`);
+    }
+  });
+
+  // afterEach(() => {
+  //   cleanDir(BASE_TEMP_DIR);
+  // });
+
+  it("generates multiple sprite files correctly", { timeout: 30_000 }, async () => {
+    const prefix = getTempPrefix(nanoid());
+
+    const outputPath = await generateStoryboards(
+      TEST_VIDEO,
+      prefix,
+      2,
+    );
+
+    const spriteFiles = fs.readdirSync(outputPath).filter(f => f.endsWith(".webp"));
+
+    expect(spriteFiles.length).greaterThan(1);
+  });
+
+});

services/our/src/tests/storyboard.integration.test.ts

@@ -0,0 +1,42 @@
+import { describe, it, expect, beforeAll, afterAll } from "vitest";
+import { existsSync, readFile, remove } from "fs-extra";
+import { join } from "node:path";
+import { generateStoryboard } from "../utils/storyboard";
+import { env } from "../config/env";
+import logger from "../utils/logger";
+import { create } from 'slvtt';
+
+// Sample video for test (you should keep a very small test MP4 under `__fixtures__`)
+const testVideoPath = join(__dirname, "fixtures", "sample.mp4");
+
+let imagePath: string;
+let vttPath: string;
+
+describe("generateStoryboard", () => {
+  it("should generate a storyboard image and a VTT file", async () => {
+    const result = await generateStoryboard(testVideoPath);
+
+    imagePath = result.imagePath;
+    vttPath = result.vttPath;
+
+    // Check that both files exist
+    expect(existsSync(imagePath)).toBe(true);
+    expect(existsSync(vttPath)).toBe(true);
+
+    logger.info(`imagePath=${imagePath}, vttPath=${vttPath}`)
+
+    // Check contents of VTT file
+    const vttContent = await readFile(vttPath, { encoding: "utf-8" }) as string;
+    logger.info(`vttContent=${vttContent}`)
+
+    expect(vttContent.startsWith("WEBVTT")).toBe(true);
+    expect(vttContent).toContain("-->"); // should have timestamps
+    expect(vttContent).toMatch(/#xywh=\d+,\d+,\d+,\d+/); // should have sprite metadata
+  }, 35_000);
+
+  // afterAll(async () => {
+  //   // Clean up generated files after test
+  //   if (imagePath) await remove(imagePath);
+  //   if (vttPath) await remove(vttPath);
+  // });
+});

services/our/src/tests/transcription.integration.test.ts

@@ -0,0 +1,61 @@
+
+import { describe, it, expect, beforeAll } from 'vitest';
+import { join, extname } from 'node:path';
+import { stat, readFile } from 'node:fs/promises';
+import { extractAudio, generateClosedCaptions } from '../utils/transcription';
+import logger from '../utils/logger';
+
+const sampleVideoPath = join(__dirname, 'fixtures', 'sample-projektmelody-chaturbate-2025-07-21.mp4');
+const expectedPhrases = [
+  'WEBVTT',
+  `I didn't see that, I didn't know that.`,
+  `Oh, I don't have to use olives, fantastic.`,
+  `Good, I like that I have choices here.`,
+  `Same comes to this.`,
+  `In anything in the world, life is so scary.`,
+  `You are flying through space on this little blue ball`,
+  `in the universe, there's so much in this world`,
+  `that you don't have control of.`,
+  `But when it comes to the world,`
+]
+
+
+describe('generateClosedCaptions', () => {
+
+  let vttOutputPath: string
+  let outputText: string
+
+  beforeAll(async () => {
+    vttOutputPath = await generateClosedCaptions(sampleVideoPath)
+    logger.info(`generateClosedCaptions finished with vttOutputPath=${vttOutputPath}`)
+    outputText = await readFile(vttOutputPath, { encoding: 'utf-8' })
+  }, 20_000)
+
+  it('should contain expected phrases', async () => {
+    for (const phrase of expectedPhrases) {
+      expect(outputText).toContain(phrase)
+    }
+
+  })
+
+
+})
+
+
+describe('extractAudio', () => {
+  let audioFilePath: string
+
+  beforeAll(async () => {
+    audioFilePath = await extractAudio(sampleVideoPath)
+  }, 20_000)
+
+  it('should be a wav file', () => {
+    expect(extname(audioFilePath)).toBe('.wav')
+  })
+
+  it('should exist on disk', async () => {
+    const stats = await stat(audioFilePath)
+    expect(stats.isFile()).toBe(true)
+    expect(stats.size).toBeGreaterThan(0)
+  })
+})

services/our/src/utils/cache.ts

@@ -7,6 +7,7 @@ import { readdir, unlink } from 'fs/promises';
 import { env } from '../config/env';
 import Keyv from 'keyv';
 import KeyvPostgres from "@keyv/postgres"
+import logger from './logger';
 
 const keyv = new Keyv(new KeyvPostgres({ uri: env.DATABASE_URL, schema: 'keyv' }));
 keyv.on('error', (err) => {
@@ -26,7 +27,7 @@ const MAX_RETRIES = 10; // Max retries to acquire lock
 
 
 export async function getOrDownloadAsset(client: S3Client, bucket: string, key: string): Promise<string> {
-  console.log(`getOrDownloadAsset with bucket=${bucket} key=${key}`)
+  logger.debug(`getOrDownloadAsset with bucket=${bucket} key=${key}`)
 
   if (!client) throw new Error('getOrDownloadAsset requires S3Client as first argument');
   if (!bucket) throw new Error('getOrDownloadAsset requires bucket as second argument');
@@ -115,7 +116,7 @@ export async function cleanExpiredFiles(): Promise<number> {
     const key = file.replace(cacheDir, '');
     const cacheKey = `${bucket}:${key}`;
     const fullPath = join(cacheDir, file);
-    // console.log(`file=${file} key=${key} cacheKey=${cacheKey}`)
+    logger.debug(`file=${file} key=${key} cacheKey=${cacheKey}`)
     if (file === bucket) continue;
 
     const stillCached = await cache.get(cacheKey);
@@ -123,7 +124,7 @@ export async function cleanExpiredFiles(): Promise<number> {
       try {
         rmSync(fullPath, { recursive: true, force: true });
         deletedCount++;
-        console.log(`Deleted expired file: ${fullPath}`);
+        logger.debug(`Deleted expired file: ${fullPath}`);
       } catch (err) {
         console.warn(`Failed to delete file ${fullPath}:`, err);
       }

services/our/src/utils/logger.ts

@@ -0,0 +1,36 @@
+import pino from 'pino'
+import { env } from '../config/env'
+
+let hooks
+const isTest = env.NODE_ENV === 'test'
+
+if (env.NODE_ENV === 'test') {
+  const { prettyFactory } = require('pino-pretty')
+  const prettify = prettyFactory({ sync: true, colorize: true })
+  hooks = {
+    streamWrite: (s: string) => {
+      console.log(prettify(s)) // Mirror to console.log during tests. @see https://github.com/pinojs/pino/issues/2148
+      return s
+    },
+  }
+}
+
+const isProd = env.NODE_ENV === 'production'
+const logger = pino({
+  level: env.LOG_LEVEL,
+  ...(isTest && { hooks }),
+  ...(isProd
+    ? {}
+    : {
+      transport: {
+        target: 'pino-pretty',
+        options: {
+          colorize: true,
+          sync: true,
+        },
+      },
+    }),
+})
+
+
+export default logger

services/our/src/utils/python.ts

@@ -7,64 +7,42 @@ import { existsSync } from "fs-extra";
 export async function preparePython() {
   const spawn = await getNanoSpawn();
   const venvPath = env.VENV;
+  const venvBin = join(venvPath, "bin");
 
-  // Determine Python executable
+  // 1. Locate python3
   let pythonCmd;
   try {
     pythonCmd = which.sync("python3");
   } catch {
-    console.error("Python is not installed or not in PATH.");
     throw new Error("Python not found in PATH.");
   }
 
-  // If venv doesn't exist, create it
+  // 2. Create venv if missing
   if (!existsSync(venvPath)) {
-    console.error("Python venv not found. Creating one...");
+    console.log("Creating Python venv...");
-
+    await spawn(pythonCmd, ["-m", "venv", venvPath], {
-    try {
+      cwd: env.APP_DIR,
-      await spawn(pythonCmd, ["-m", "venv", venvPath], {
+    });
-        cwd: env.APP_DIR,
+    console.log("✅ Python venv created.");
-      });
-
-      console.log("Python venv successfully created.");
-    } catch (err) {
-      console.error("Failed to create Python venv:", err);
-      throw new Error(
-        "Python venv creation failed. Check if python3 and python3-venv are installed."
-      );
-    }
   } else {
     console.log("Using existing Python venv.");
   }
 
-  // Activate pip in the venv
+  // 3. Install requirements.txt
-  const pipCmd = join(venvPath, "bin", "pip");
+  const pipCmd = join(venvBin, "pip");
+  console.log("Installing requirements.txt...");
+  await spawn(pipCmd, ["install", "-r", "requirements.txt"], {
+    cwd: env.APP_DIR,
+  });
+  console.log("✅ requirements.txt installed.");
 
-  // Check if YOLO exists (example: checking if 'yolo' package is installed)
+  // 4. Confirm vcsi CLI binary exists
-  // This check can be customized to your specific condition (e.g., check a file or run `pip show yolo`)
+  const vcsiBinary = join(venvBin, "vcsi");
-  let yoloExists = false;
+  if (!existsSync(vcsiBinary)) {
-  try {
+    console.error("❌ vcsi binary not found in venv after installing requirements.");
-    // Run `pip show ultralytics` or your yolo package name to check if installed
+    console.error("Make sure 'vcsi' is listed in requirements.txt and that it installs a CLI.");
-    await spawn(pipCmd, ["show", "ultralytics"], { cwd: env.APP_DIR });
+    throw new Error("vcsi installation failed or did not expose CLI.");
-    yoloExists = true;
-  } catch {
-    yoloExists = false;
   }
 
-  if (!yoloExists) {
+  console.log("✅ vcsi CLI is available at", vcsiBinary);
-    console.log("YOLO not found in venv. Installing requirements.txt...");
-
-    try {
-      // Install requirements.txt using pip inside venv
-      await spawn(pipCmd, ["install", "-r", "requirements.txt"], {
-        cwd: env.APP_DIR,
-      });
-      console.log("✅ requirements.txt installed successfully.");
-    } catch (err) {
-      console.error("Failed to install requirements.txt:", err);
-      throw new Error("requirements.txt installation failed.");
-    }
-  } else {
-    console.log("YOLO detected in venv, skipping requirements installation.");
-  }
 }

services/our/src/utils/s3.ts

@@ -5,6 +5,7 @@ import {
 } from "@aws-sdk/client-s3"; // @see https://www.backblaze.com/docs/cloud-storage-use-the-aws-sdk-for-javascript-v3-with-backblaze-b2
 import { readFile } from 'fs/promises';
 import { env } from '../config/env' // adjust this path based on your project structure
+import logger from "./logger";
 
 let client: S3Client | null = null
 
@@ -21,7 +22,7 @@ export async function uploadFile(
     if (!key) throw new Error('uploadFile requires key as third param');
     if (!filePath) throw new Error('uploadFile requires filePath as fourth param');
     if (!mimetype) throw new Error('uploadFile requires mimetype as fifth param');
-    console.log(`uploadFile filePath=${filePath} with key=${key} bucket=${bucket}`);
+    logger.debug(`[uploadFile] filePath=${filePath} with key=${key} bucket=${bucket}`);
 
     try {
         // Read file content from disk
@@ -41,11 +42,11 @@ export async function uploadFile(
 
     } catch (caught) {
         if (caught instanceof S3ServiceException) {
-            console.error(
+            logger.error(
                 `Error from S3 while uploading to ${bucket}. ${caught.name}: ${caught.message}`,
             );
         } else {
-            console.error(`Unexpected error during upload:`, caught);
+            logger.error(`Unexpected error during upload:`, caught);
         }
 
         throw new Error(`Failed to upload ${filePath} to s3://${bucket}/${key}`);

services/our/src/utils/storyboard2.ts

@@ -0,0 +1,142 @@
+// storyboard2.ts
+// ported from https://github.com/Dibyakshu/go-video-storyboard-generator/blob/main/main.go
+
+import { existsSync, mkdirSync, readdirSync, rmSync, writeFileSync } from 'fs';
+import { join } from 'path';
+import { getNanoSpawn } from './nanoSpawn';
+
+// Entry point
+(async function main() {
+  if (process.argv.length < 5) {
+    showUsage();
+  }
+
+  const fps = parseInt(process.argv[2], 10);
+  const inputFile = process.argv[3];
+  const outputDir = process.argv[4];
+
+  if (isNaN(fps) || fps <= 0) {
+    console.error('Error: Invalid FPS value. Please provide a positive integer.');
+    process.exit(1);
+  }
+
+  ensureDirectoryExists(outputDir);
+
+  const startTime = Date.now();
+  const thumbnailsDir = join(outputDir, 'thumbnails');
+  ensureDirectoryExists(thumbnailsDir);
+
+  generateThumbnails(fps, inputFile, thumbnailsDir);
+  const storyboardImage = join(outputDir, 'storyboard.jpg');
+  generateStoryboard(thumbnailsDir, storyboardImage);
+  const vttFile = join(outputDir, 'storyboard.vtt');
+  generateVTT(fps, thumbnailsDir, vttFile);
+  cleanup(thumbnailsDir);
+
+  const elapsed = (Date.now() - startTime) / 1000;
+  console.log(`Process completed in ${elapsed.toFixed(2)} seconds.`);
+})();
+
+function showUsage(): never {
+  console.log('Usage: ts-node storyboard.ts <fps> <input_file> <output_dir>');
+  console.log('Example: ts-node storyboard.ts 1 example.mp4 ./output');
+  process.exit(1);
+}
+
+function ensureDirectoryExists(path: string) {
+  if (!existsSync(path)) {
+    mkdirSync(path, { recursive: true });
+  }
+}
+
+export async function generateThumbnails(fps: number, inputFile: string, thumbnailsDir: string) {
+  const spawn = await getNanoSpawn()
+  console.log('Generating thumbnails...');
+  try {
+    spawn('ffmpeg', [
+      '-i', inputFile,
+      '-vf', `fps=1/${fps},scale=384:160`,
+      join(thumbnailsDir, 'thumb%05d.jpg'),
+    ], { stdio: 'inherit' });
+    console.log('Thumbnails generated successfully.');
+  } catch (err) {
+    console.error('Error generating thumbnails:', err);
+    process.exit(1);
+  }
+}
+
+export async function generateStoryboard(thumbnailsDir: string, storyboardImage: string) {
+  const spawn = await getNanoSpawn()
+  console.log('Creating storyboard.jpg...');
+  const thumbnails = readdirSync(thumbnailsDir).filter(f => f.endsWith('.jpg'));
+  if (thumbnails.length === 0) {
+    console.error('No thumbnails found. Exiting.');
+    process.exit(1);
+  }
+
+  const columns = 10;
+  const rows = Math.ceil(thumbnails.length / columns);
+  const tileFilter = `tile=${columns}x${rows}`;
+
+  try {
+    await spawn('ffmpeg', [
+      '-pattern_type', 'glob',
+      '-i', join(thumbnailsDir, '*.jpg'),
+      '-filter_complex', tileFilter,
+      storyboardImage,
+    ], { stdio: 'inherit' });
+    console.log('Storyboard image created successfully.');
+  } catch (err) {
+    console.error('Error creating storyboard image:', err);
+    process.exit(1);
+  }
+}
+
+function generateVTT(fps: number, thumbnailsDir: string, vttFile: string) {
+  console.log('Generating storyboard.vtt...');
+  const thumbnails = readdirSync(thumbnailsDir).filter(f => f.startsWith('thumb') && f.endsWith('.jpg'));
+  const durationPerThumb = fps;
+  let vttContent = 'WEBVTT\n\n';
+
+  for (let i = 0; i < thumbnails.length; i++) {
+    const start = i * durationPerThumb * 1000;
+    const end = start + durationPerThumb * 1000;
+
+    const x = (i % 10) * 384;
+    const y = Math.floor(i / 10) * 160;
+
+    vttContent += `${formatDuration(start)} --> ${formatDuration(end)}\n`;
+    vttContent += `https://insertlinkhere.com/storyboard.jpg#xywh=${x},${y},384,160\n\n`;
+  }
+
+  try {
+    writeFileSync(vttFile, vttContent, 'utf8');
+    console.log('Storyboard VTT file generated successfully.');
+  } catch (err) {
+    console.error('Error writing VTT file:', err);
+    process.exit(1);
+  }
+}
+
+function cleanup(thumbnailsDir: string) {
+  console.log('Cleaning up temporary files...');
+  try {
+    rmSync(thumbnailsDir, { recursive: true, force: true });
+  } catch (err) {
+    console.error('Error cleaning up thumbnails:', err);
+    process.exit(1);
+  }
+}
+
+function formatDuration(ms: number): string {
+  const totalSeconds = Math.floor(ms / 1000);
+  const hours = Math.floor(totalSeconds / 3600);
+  const minutes = Math.floor((totalSeconds % 3600) / 60);
+  const seconds = totalSeconds % 60;
+  const milliseconds = ms % 1000;
+  return `${pad(hours)}:${pad(minutes)}:${pad(seconds)}.${pad(milliseconds, 3)}`;
+}
+
+function pad(n: number, width = 2): string {
+  return n.toString().padStart(width, '0');
+}

services/our/src/utils/transcription.ts

@@ -0,0 +1,85 @@
+import { getNanoSpawn } from "./nanoSpawn";
+import { SubprocessError } from "nano-spawn";
+import { mkdir } from "fs-extra";
+import { join, extname, basename, dirname } from "node:path";
+import { env } from "../config/env";
+import { nanoid } from "nanoid";
+import logger from "./logger";
+
+// @see https://github.com/ggml-org/whisper.cpp/tree/master#quick-start
+// ffmpeg -i input.mp4 -vn -ar 16000 -ac 1 -c:a pcm_s16le output.wav
+export async function extractAudio(videoFilePath: string): Promise<string> {
+  if (!videoFilePath) throw new Error('extractAudio was missing first arg videoFilePath');
+  const spawn = await getNanoSpawn();
+  const tmpFile = join(env.CACHE_ROOT, `${nanoid()}.wav`);
+  logger.info(`tmpFile=${tmpFile}`)
+
+  // Make sure CACHE_ROOT exists
+  await mkdir(env.CACHE_ROOT, { recursive: true });
+
+  try {
+    await spawn('ffmpeg', [
+      '-i', videoFilePath,
+      '-vn',
+      '-ar', '16000',
+      '-ac', '1',
+      '-c:a', 'pcm_s16le',
+      tmpFile,
+    ]);
+  } catch (err) {
+    throw new Error(`Failed to extract audio from ${videoFilePath}: ${(err as Error).message}`);
+  }
+
+  return tmpFile;
+}
+
+
+// @see https://github.com/ggml-org/whisper.cpp/blob/master/.devops/main.Dockerfile
+// @see https://github.com/ggml-org/whisper.cpp/tree/master
+export async function generateClosedCaptions(filePath: string): Promise<string> {
+  const spawn = await getNanoSpawn()
+
+
+  const ext = extname(filePath).toLowerCase()
+  let wavFilePath = filePath
+
+  if (ext !== '.wav') {
+    wavFilePath = await extractAudio(filePath)
+  }
+
+  const tmpVttFile = join(env.CACHE_ROOT, `${nanoid()}.vtt`)
+  const tmpVttFileNoExt = join(dirname(tmpVttFile), basename(tmpVttFile, '.vtt'))
+  logger.info(`let us create tmpVttFile=${tmpVttFile} tmpVttFileNoExt=${tmpVttFileNoExt}`)
+
+  try {
+    logger.info('we are calling whisper-cli now.');
+    const { stdout, stderr } = await spawn('whisper-cli', [
+      '--file', wavFilePath,
+      '--output-vtt',
+      '--output-file', tmpVttFileNoExt, // whisper-cli appends .vtt to whatever filename we give it.
+    ], {
+      cwd: env.WHISPER_DIR
+    });
+
+    logger.info('we just finished calling whisper-cli.');
+    logger.info({ stdout, stderr });
+
+  } catch (err) {
+    logger.error('CAUGHT an error:', err);
+
+    if (err instanceof SubprocessError) {
+      logger.error(`whisper-cli failed with exit code ${err.exitCode}`);
+      logger.error(`stdout: ${err.stdout}`);
+      logger.error(`stderr: ${err.stderr}`);
+      logger.error(`command: ${err.command}`);
+    } else {
+      logger.error('Unexpected error:');
+      logger.error(err)
+    }
+
+    throw new Error(`Failed to generate closed captions from ${wavFilePath}: ${(err as Error).message}`);
+  }
+
+  logger.info(`>>>>>>>>>>>>>>>>>> ${tmpVttFile} should exist`)
+  return tmpVttFile
+}

services/our/src/views/partials/navbar.hbs

@@ -11,6 +11,7 @@
       <li><a href="/vods">VODs</a></li>
       <li><a href="/streams"><s>🚧 Streams</s></a></li>
       <li><a href="/vtubers">VTubers</a></li>
+      <li><a href="/pricing">Pricing</a></li>
       {{#if (hasRole "supporterTier1" "moderator" "admin" user)}}
       <li><a href="/uploads">Uploads</a></li>
       {{/if}}

services/our/src/views/pricing.hbs

@@ -0,0 +1,98 @@
+{{#> main}}
+
+<header class="container">
+  {{> navbar}}
+</header>
+
+<main class="container pico">
+  <section id="pricing">
+    <h2>Pricing</h2>
+
+    <p>future.porn is free to use, but to keep the site running, we need your help! In return, we offer extra perks
+      to supporters.</p>
+
+    <table>
+      <thead>
+        <tr>
+          <th>Feature</th>
+          <th>User</th>
+          <th>Supporter Tier 1</th>
+          <th>Supporter Tier 6</th>
+        </tr>
+      </thead>
+      <tbody>
+        <tr>
+          <td>View</td>
+          <td>✔️</td>
+          <td>✔️</td>
+          <td>✔️</td>
+        </tr>
+        <tr>
+          <td>Torrent Downloads</td>
+          <td>✔️</td>
+          <td>✔️</td>
+          <td>✔️</td>
+        </tr>
+        <tr>
+          <td>CDN Downloads</td>
+          <td></td>
+          <td>✔️</td>
+          <td>✔️</td>
+        </tr>
+        <tr>
+          <td>Ad-Free</td>
+          <td></td>
+          <td>✔️</td>
+          <td>✔️</td>
+        </tr>
+        <tr>
+          <td>Upload</td>
+          <td></td>
+          <td>✔️</td>
+          <td>✔️</td>
+        </tr>
+        <tr>
+          <td><abbr title="Sex toy playback syncronization">Funscripts</abbr></td>
+          <td></td>
+          <td>✔️</td>
+          <td>✔️</td>
+        </tr>
+        <tr>
+          <td>Closed Captions</td>
+          <td></td>
+          <td>✔️</td>
+          <td>✔️</td>
+        </tr>
+        <tr>
+          <td><abbr title="Closed Captions">CC</abbr> Search</td>
+          <td></td>
+          <td>✔️</td>
+          <td>✔️</td>
+        </tr>
+        <tr>
+          <td>CSV Export</td>
+          <td></td>
+          <td></td>
+          <td>✔️</td>
+        </tr>
+        <tr>
+          <td>SQL Export</td>
+          <td></td>
+          <td></td>
+          <td>✔️</td>
+        </tr>
+        <tr>
+          <td>vibeui PyTorch Model</td>
+          <td></td>
+          <td></td>
+          <td>✔️</td>
+        </tr>
+      </tbody>
+    </table>
+
+  </section>
+
+  {{> footer}}
+</main>
+
+{{/main}}

services/our/src/views/profile.hbs

@@ -13,6 +13,9 @@
     <p><strong>Identicon:</strong> {{{identicon user.id 48}}}</p>
     <p><strong>Roles:</strong> {{#each user.roles}}{{this.name}}{{#unless @last}}, {{/unless}}{{/each}}
     </p>
+    <p><strong>Perks:</strong> @todo
+      {{!-- @todo {{#each user.perks}}{{this.name}}{{#unless @last}}, {{/unless}}{{/each}} --}}
+    </p>
   </section>
 
   <a href="/logout">Logout</a>

services/our/src/views/vod.hbs

@@ -1,7 +1,10 @@
 {{#> main}}
 <!-- Header -->
 <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/video.js@8.23.3/dist/video-js.min.css">
-
+<link rel="stylesheet"
+  href="https://cdn.jsdelivr.net/npm/videojs-vtt-thumbnails@0.0.13/dist/videojs-vtt-thumbnails.min.css">
+<link rel="stylesheet"
+  href="https://cdn.jsdelivr.net/gh/Teyuto/videojs-vtt-thumbnails@main/src/videojs-vtt-thumbnails.min.css">
 <style>
   .vjs-funscript-indicator {
     color: #0f0;
@@ -46,6 +49,7 @@
       <video id="player" class="video-js vjs-fluid" controls preload="auto" poster="{{getCdnUrl vod.thumbnail}}"
         data-setup='{}' data-playlist="{{signedHlsUrl vod.hlsPlaylist}}">
         <source src="/hls/{{vod.id}}/master.m3u8" type="application/x-mpegURL">
+        <track kind="captions" src="{{getCdnUrl vod.asrVttKey}}" srclang="en" label="English" default>
         <p class="vjs-no-js">
           To view this video please enable JavaScript, and consider upgrading to a
           web browser that
@@ -191,8 +195,34 @@
       {{/if}}
 
 
+      <h3>Closed Captions / Subtitles</h3>
+      {{#if vod.asrVttKey}}
+      <a id="asr-vtt" data-url="{{getCdnUrl vod.asrVttKey}}" data-file-name="{{basename vod.asrVttKey}}"
+        x-on:click.prevent="download($el.dataset.url, $el.dataset.fileName)" href="{{getCdnUrl vod.asrVttKey}}"
+        alt="Closed Captions VTT file">{{icon "download" 24}} Closed Captions</a>
+      {{else}}
+      <article>
+        Closed captions are processing.
+      </article>
+      {{/if}}
+
+
+
+
 
       {{#if (isModerator user)}}
+
+      <h3>Storyboard Images</h3>
+      {{#if vod.slvttVTTKey}}
+      <a id="slvtt" data-url="{{getCdnUrl vod.slvttVTTKey}}" data-file-name="{{basename vod.slvttVTTKey}}"
+        x-on:click.prevent="download($el.dataset.url, $el.dataset.fileName)" href="{{getCdnUrl vod.slvttVTTKey}}"
+        alt="slvttVTTKey">{{icon "download" 24}} slvttVTTKey</a>
+      {{else}}
+      <article>
+        Storyboard Images are processing
+      </article>
+      {{/if}}
+
       <h2>Moderator Controls</h2>
       <button hx-post="/vods/{{vod.id}}/process" hx-target="body">{{icon "processing" 24}} Re-Schedule Vod
         Processing</button>
@@ -210,13 +240,10 @@
 
   {{>footer}}
 </main>
-{{!-- <script src=" https://cdn.jsdelivr.net/npm/video.js@8.23.3/dist/video.min.js "></script> --}}
+<script src=" https://cdn.jsdelivr.net/npm/video.js@8.23.3/dist/video.min.js "></script>
 <script>
 
 
-  //var player = videojs('vid1', {
-  //fluid: true
-  //});
 
   async function download(cdnUrl, fileName) {
 
@@ -242,26 +269,68 @@
 
 
 </script>
-<script src="https://cdn.jsdelivr.net/npm/video.js@8.23.3/dist/video.min.js"></script>
+<script src="https://cdn.jsdelivr.net/gh/Teyuto/videojs-vtt-thumbnails@main/src/videojs-vtt-thumbnails.min.js"></script>
+{{!-- <script src="https://cdn.jsdelivr.net/npm/video.js@8.23.3/dist/video.min.js"></script> --}}
+{{!-- <script src="https://cdn.jsdelivr.net/npm/videojs-vtt-thumbnails@0.0.13/dist/videojs-vtt-thumbnails.min.js"></script> --}}
+{{!-- <script type="module">
+  import { default as caca } from "https://esm.sh/gh/mayeaux/videojs-vtt-thumbnails@260a63a/es2022/videojs-vtt-thumbnails.mjs";
+  console.log('caca as follows')
+  console.log(caca)
+  caca()
+</script> --}}
+
+{{!-- 
+  Script 4: Initialize the buttplug plugin and pass in the funscript URL from the DOM
+--}}
+<script type="module">
+  //import videojs from 'https://cdn.jsdelivr.net/npm/video.js@8/+esm'
+  //import 'https://esm.sh/gh/mayeaux/videojs-vtt-thumbnails';
+
+  const player = videojs('#player');
+  const funscriptElement = document.querySelector('#funscript');
+
+  //if (funscriptElement) {
+  //  player.buttplug({
+  //    funscriptUrl: funscriptElement.dataset.url
+  //  });
+  //} else {
+  //  console.error('Element with id "funscript" not found.');
+  //}
+
+  //console.log('vtt-thumbnails')
+  //console.log(player.vttThumbnails)
+  // Initialize videojs-vtt-thumbnails
+  //player.vttThumbnails({
+  //  src: "{{getCdnUrl vod.slvttVTTKey}}",
+  //  showTimestamp: true
+  //});
+  player.vttThumbnails({
+    //spriteUrl: 'path/to/sprite.jpg',
+    vttData: {
+      url: '{{getCdnUrl vod.slvttVTTKey}}'
+    }
+  });
+
+</script>
 
 {{!-- 
   Script 1: Load Buttplug.js from Skypack CDN and expose it to window.buttplug
 --}}
-<script type="module">
+{{!-- <script type="module">
   import {
     ButtplugClient,
     ButtplugBrowserWebsocketClientConnector
   } from 'https://cdn.skypack.dev/buttplug';
 
   window.buttplug = { ButtplugClient, ButtplugBrowserWebsocketClientConnector };
-</script>
+</script> --}}
 
 
 
 {{!-- 
   Script 2: Define reusable utility components for funscript and buttplug indicators 
 --}}
-<script type="module">
+{{!-- <script type="module">
   const Plugin = videojs.getPlugin('plugin');
 
   const createIndicator = (Component, className, defaultText) => {
@@ -283,13 +352,13 @@
     'ButtplugIndicator',
     createIndicator(videojs.getComponent('Component'), 'vjs-buttplug-indicator', 'Buttplug.js not connected')
   );
-</script>
+</script> --}}
 
 
 {{!-- 
   Script 3: Main ButtplugPlugin class — handles connection, syncing, and device control 
 --}}
-<script type="module">
+{{!-- <script type="module">
   class ButtplugPlugin extends videojs.getPlugin('plugin') {
     constructor(player, options) {
       super(player, options);
@@ -481,22 +550,10 @@
   }
 
   videojs.registerPlugin('buttplug', ButtplugPlugin);
-</script>
+</script> --}}
+
+
 
-{{!-- 
-  Script 4: Initialize the plugin and pass in the funscript URL from the DOM
---}}
-<script type="module">
-  const player = videojs('#player');
-  const funscriptElement = document.querySelector('#funscript');
 
-  if (funscriptElement) {
-    player.buttplug({
-      funscriptUrl: funscriptElement.dataset.url
-    });
-  } else {
-    console.error('Element with id "funscript" not found.');
-  }
-</script>
 
 {{/main}}

services/scout/README.md

@@ -12,11 +12,17 @@
 #### Alby Event numbers and their meanings
 
 Action 0: heartbeat
-Action 4:
+Action 2: connection request
+Action 4: connection acknowledgment
 Action 10: Presence?
 Action 11:
 Action 15: Broadcast
 
 ### Ideas
 
-There is the url `https://chaturbate.com/api/public/affiliates/onlinerooms/?wm=DiPkB&client_ip=request_ip` which shows all live channels. We could query this.
+There is the url `https://chaturbate.com/api/public/affiliates/onlinerooms/?wm=DiPkB&client_ip=request_ip` which shows all live channels. We could query this.
+
+
+### Troubleshooting
+
+#### Lots of `{"action":4}` Getting lots of Action 4 frames? Might be rate limiting or something? Simple fix is to reboot PC.