# Utility Bill Agent — Skill File for Codex CLI This file teaches Codex CLI how to build a Playwright-based agent that: 1. Logs into a utility provider portal 2. Scrapes billing data and downloads new bill PDFs 3. Emails each PDF to a PM software's bill-entry address 4. Optionally compares bill amounts to historical averages and flags anomalies 5. Tracks processed bills so reruns don't duplicate 6. Runs daily on a schedule It is the distilled output of building two of these agents (MidAmerican Energy and Xcel Energy) and hitting every common failure mode. --- ## Project structure ``` bill-agent/ src/ index.js # Entry point. CLI flags: --headed, --dry-run, --skip-email auth.js # Browser launch, login, session reuse, modal/cookie dismissal scraper.js # Navigate portal, scrape billing tables, handle pagination downloader.js # Download bill PDFs (via download events or direct HTTP fetch) pdf-parser.js # Extract fields from PDFs (pdftotext or pdf-parse + regex) anomaly.js # Compare current charges to historical average, flag outliers mailer.js # Send PDFs to PM software via Resend state.js # JSON state file for deduplication + history config.js # Load .env, export config object logger.js # Winston logger, console transport data/ state.json # Tracks processed bills per account auth-state.json # Playwright session cookies (delete to force re-login) bills/ # Downloaded PDFs cron.log # Scheduled run output .env # Credentials and config (NEVER commit this) package.json # Node ESM ("type": "module") ``` --- ## Dependencies ```json { "dotenv": "^16.4.7", "pdf-parse": "^1.1.1", "playwright": "^1.49.1", "resend": "^6.10.0", "winston": "^3.17.0" } ``` After `npm install`, run `npx playwright install chromium`. --- ## Codex CLI operating notes Codex CLI runs inside the user's local project folder. It can inspect files, edit files, and run terminal commands, but it may ask the user to approve file changes or command execution depending on the current approval and sandbox settings. For this project, prefer small, verifiable steps: 1. Inspect the spec, screenshots, and existing files. 2. Create the project structure. 3. Install dependencies. 4. Run one headed dry run. 5. Fix one failure at a time. When you need to run a command, show the user what you are about to run in plain language. If Codex asks for approval, tell the user whether the action is normal and safe for this project. Normal approvals include editing files inside `bill-agent/`, running `npm install`, running `npx playwright install chromium`, and running `node src/index.js --headed --dry-run`. Do not ask the user to approve destructive commands such as deleting unrelated folders, changing system security settings, or modifying files outside the project unless there is a clear reason and the user understands it. Codex CLI reads files from the current working directory. Do not assume the user can drag files into the terminal. Tell the user to put `my-bill-agent-spec.md`, `utility-bill-agent-skill-codex.md`, and the `frames/` folder inside the same `bill-agent/` folder before starting. --- ## Critical patterns ### 1. Always run headed first ```bash node src/index.js --headed --dry-run ``` Never assume a scraper works headless until you've watched it run headed at least three times successfully. Use `--dry-run` to skip downloads and emails while debugging. Use `--skip-email` to test downloads without actually sending. ### 2. Login and session management - Store Playwright browser context (cookies) to `data/auth-state.json` after successful login. - On startup, try to reuse saved session by navigating to a protected page. - If the session is expired (redirected to login), re-authenticate. - After login, save the new session immediately. - Handle cookie banners and modals after EVERY navigation (they often reappear). ```js // Pattern: session check const response = await page.goto(PROTECTED_URL, { waitUntil: 'domcontentloaded' }); const currentUrl = page.url(); if (currentUrl.includes('login') || currentUrl.includes('Login') || currentUrl.includes('signin')) { // Session expired, need to re-login } ``` ### 3. Cookie banners and intrusive modals Cookie banners and popup modals are the #1 source of scraper failures. They block interaction with page elements underneath. - Dismiss cookie banners after EVERY `page.goto()` and after pagination clicks. - Some banners use `position: fixed`, which makes Playwright report them as not visible. - When Playwright can't click a banner, use `page.evaluate()` with raw DOM JS. - For modals asking to enroll in paperless billing, change settings, or take any action: NEVER click accept/enroll. Only click dismiss/close/no thanks. ```js // Pattern: dismiss cookie banner via raw JS (bypasses visibility checks) async function dismissCookieBanner(page) { try { await page.evaluate(() => { // Try common cookie accept selectors const candidates = [ '#cookie-accept-button', '#onetrust-accept-btn-handler', '[id*="cookie"][id*="accept"]', 'button[aria-label*="Accept"]', ]; for (const sel of candidates) { const btn = document.querySelector(sel); if (btn) { btn.click(); return; } } }); } catch (e) { /* banner not present, that's fine */ } } // Pattern: dismiss modal popups via STRICT ALLOWLIST of safe button text // Only click buttons whose text matches one of these patterns. Never click anything else. const SAFE_DISMISS_TEXTS = [ /^no thanks$/i, /^not now$/i, /^maybe later$/i, /^skip$/i, /^close$/i, /^dismiss$/i, /^cancel$/i, /^×$/, ]; async function dismissModals(page) { const buttons = page.locator('button:visible, a:visible[role="button"]'); const count = await buttons.count(); for (let i = 0; i < count; i++) { const btn = buttons.nth(i); const text = (await btn.innerText().catch(() => '')).trim(); if (SAFE_DISMISS_TEXTS.some(re => re.test(text))) { await btn.click().catch(() => {}); await page.waitForTimeout(300); } } } ``` ### 4. Salesforce Lightning portals (shadow DOM) Some utility portals (e.g., Xcel Energy) use Salesforce Lightning, which renders components inside shadow DOM. - `document.querySelectorAll()` inside `page.evaluate()` returns nothing (can't pierce shadow boundaries). - Playwright locators (`page.locator()`) pierce shadow DOM natively. - Always prefer Playwright locators over `page.evaluate()` for element interaction. - Use `page.evaluate()` only for reading light-DOM data like `document.body.innerText`. ```js // WRONG: finds 0 elements in Salesforce Lightning const rows = await page.evaluate(() => document.querySelectorAll('tr')); // RIGHT: Playwright pierces shadow DOM const rows = page.locator('tr'); const count = await rows.count(); ``` ### 5. Login forms with hidden duplicate fields (Gigya, etc.) Some portals use form-rendering frameworks (Gigya is a common one) that render the same login form multiple times invisibly. Playwright's strict mode will throw "6 elements found matching" errors. ```js // WRONG: errors with strict-mode violation await page.locator('input[name="username"]').fill(username); // RIGHT: filter to visible elements only await page.locator('input[name="username"]:visible').first().fill(username); ``` ### 6. PDF downloads via Playwright Two patterns, depending on how the portal serves PDFs. **Pattern A: PDF triggered by clicking a button.** Set up the download event listener BEFORE triggering the download. ```js const downloadPromise = page.waitForEvent('download'); await page.locator('button:has-text("Download")').click(); const download = await downloadPromise; await download.saveAs(`data/bills/bill_${accountNumber}_${date}.pdf`); ``` **Pattern B: PDF available at a direct URL.** Many portals expose direct bill links like `/viewbill?id=...` or `/bills/`. These can be fetched via direct HTTP using the authenticated browser context. ```js // Use the authenticated context to fetch the PDF const response = await context.request.get(pdfUrl); const buffer = await response.body(); fs.writeFileSync(`data/bills/bill_${accountNumber}_${date}.pdf`, buffer); ``` Pattern B is faster and more reliable if the portal exposes direct links. Look for them. They're often in href attributes inside the bills table. ### 7. PDF text extraction Use `pdf-parse` for text extraction. Be aware that it often strips spaces: - "Account Number: 12345-67890" becomes "AccountNumber:12345-67890" - "New Charges $87.50" becomes "NewCharges$87.50" Write all regex patterns for the spaceless form. Test patterns against actual extracted text before assuming they work. ```js import pdfParse from 'pdf-parse'; import { readFileSync } from 'fs'; const buffer = readFileSync(filepath); const data = await pdfParse(buffer); const text = data.text; // spaceless // Adjust per provider — these are example patterns const newCharges = text.match(/NewCharges\$([\d,]+\.\d{2})/)?.[1]; const accountNumber = text.match(/AccountNumber:([\d]+-[\d]+)/)?.[1]; const billDate = text.match(/DateBilled:(\d{2}\/\d{2}\/\d{2})/)?.[1]; ``` ### 8. Anomaly detection: use "New Charges", not "Total Amount Due" This is a real gotcha that will produce false positives if you get it wrong. - **New Charges** = this month's actual bill amount (this is what you want) - **Total Amount Due** = running balance after payments and credits (this is NOT what you want) If the PM made a partial payment last month, Total Amount Due can be much lower than the actual bill, producing false LOW anomaly flags constantly. ```js const ratio = newCharges / averageBill; if (ratio > 1.3) flag = 'HIGH'; // 30% above average else if (ratio < 0.5) flag = 'LOW'; // 50% below average else flag = 'NORMAL'; ``` Adjust thresholds to taste. 30%/50% is a reasonable starting point. **Seasonal note for gas bills:** gas usage spikes in winter months are normal, not anomalies. If the user complains about false positives in winter, add a comparison to the same month of the prior year instead of a 6-month rolling average. ### 9. State management (deduplication) Track which bills have been processed to avoid re-downloading and re-emailing. ```json { "accounts": { "12345-67890": { "lastBillDate": "2026-04-01", "lastCharges": 47.83, "history": [ { "date": "2026-04-01", "charges": 47.83 }, { "date": "2026-03-01", "charges": 87.50 } ] } }, "lastRun": "2026-04-10T12:00:00Z" } ``` ### 10. Email delivery via Resend ```js import { Resend } from 'resend'; const resend = new Resend(process.env.RESEND_API_KEY); await resend.emails.send({ from: 'noreply@yourdomain.com', // must be a verified domain in Resend to: 'company@invoices.appfolio.com', subject: `${address} - ${providerName} Bill ${billDate}`, text: `Bill for ${address}, account ${accountNumber}, dated ${billDate}.`, attachments: [{ filename: `${providerName}_${accountNumber}_${billDate}.pdf`, content: readFileSync(pdfPath).toString('base64'), }], }); ``` **Note for Resend free tier:** until the user has verified their own sending domain, sends are limited to the user's own email address. For testing, send to the user's email. For production, the user must add and verify a domain in Resend's dashboard (DNS records). ### 11. Scheduling Two paths depending on OS. **Mac (launchd):** create a plist that calls Node directly. Do NOT wrap in a shell script — macOS blocks shell script execution from the Downloads folder. ```xml Label com.user.bill-agent ProgramArguments /opt/homebrew/bin/node src/index.js WorkingDirectory /path/to/bill-agent EnvironmentVariables PATH /opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin StartCalendarInterval Hour7 Minute0 StandardOutPath /path/to/bill-agent/data/cron.log StandardErrorPath /path/to/bill-agent/data/cron.log ``` Save to `~/Library/LaunchAgents/com.user.bill-agent.plist` and load with `launchctl load `. **Windows (Task Scheduler):** create a scheduled task that runs `node.exe src/index.js` daily at the chosen time. Use the Task Scheduler GUI for the user since the schtasks CLI is fiddly. **Important:** the user's machine must be on for the scheduled run to fire. Laptops asleep at 7 a.m. miss their window. Recommend either: - Plugging the laptop in and leaving it open at night, OR - Hosting the agent on a small cloud VM (DigitalOcean droplet, Fly.io, etc.) — but this is a more advanced setup. Don't push the user there until they ask. ### 12. Failure notifications Wrap the entire run in a try/catch. If the agent crashes, email the user with the error and stack trace. Use Resend. ```js try { await runAgent(); } catch (error) { await sendFailureEmail({ error: error.message, stack: error.stack, timestamp: new Date().toISOString(), }); process.exit(1); } ``` If multiple providers are running and only one fails, send a single digest at the end of the run rather than one email per failure. --- ## Common failures and fixes | Symptom | Cause | Fix | |---|---|---| | Scraper finds 0 elements | Shadow DOM (Salesforce Lightning) | Switch from `page.evaluate()` to Playwright locators | | Login clicks wrong field | Multiple hidden form copies (Gigya) | Add `:visible` filter to locators | | Page never loads | Cookie banner blocking | Add `dismissCookieBanner()` after every navigation | | "Download is starting" error | Download event not set up before navigation | Call `waitForEvent('download')` BEFORE `page.goto()` | | PDF regex finds nothing | pdf-parse strips spaces | Rewrite patterns for spaceless text | | "Operation not permitted" in launchd | macOS blocks shell scripts in Downloads folder | Call node directly in plist, not via .sh wrapper | | Session works once then expires | Cookies not saved after login | Save browser context to auth-state.json after login | | Wrong table matched on page | Multiple tables in DOM | Identify target table by header text, not by index | | Pagination clicks wrong link | Account numbers look like page numbers | Filter pagination links by text length and visibility | | Resend "from" domain rejected | Sending from unverified domain | Either send to the user's own email until they verify a domain, or have them verify in Resend dashboard | | Agent crashes on `net::ERR_NETWORK_CHANGED` | User's machine slept or wifi changed mid-run | Wrap navigation in retries; recommend wired connection or hosted run | | MFA prompt blocks login | Account has MFA enabled | Disable MFA on the agent's account if utility allows app passwords; otherwise pause-and-prompt is the only option | --- ## Spec template The agent expects a spec file in this shape. The `start-here.md` orchestrator helps the user produce one. If the user pasted you a partially-filled version, fill in the blanks by asking the user. ```markdown # [Provider Name] Bill Agent — Spec ## Context - Who I am: [property manager, optional company name] - What I do today: [manual workflow] - What I want automated: [the part to automate] ## Target portal - Provider name: - Login URL: - Username: - Auth type: [standard form / OAuth / Gigya / Salesforce Lightning] - MFA enabled: [yes / no] - Account structure: [single / multiple / landlord agreement with table] - Number of properties or accounts: ## Bill location - How to navigate from login to bills: - Are bill amounts visible in a table, or only in the PDF? - Is the PDF a direct URL or a JavaScript-triggered download? ## Table structure (if applicable) - Column names: - Pagination type: [server pages / "show more" button / none] - Rows per page: ## Bill PDF format - Where is "current charges" in the PDF: - Where is the account number: - Where is the service address: - Where is the bill date: ## Delivery - PM software: [AppFolio / Buildium / etc] - Bill-entry email: - Email service: Resend - From address: [user's verified domain or user's own email for testing] ## Anomaly detection (V2) - Comparison baseline: [6-month rolling average / same month prior year] - HIGH threshold: 30% above - LOW threshold: 50% below ## Scheduling - Frequency: [daily / weekly] - Time of day: - Failure notification email: ``` --- ## Final notes for Codex CLI - **Always run headed for the first three runs.** Watch what's happening. Don't trust headless until you've trusted headed. - **Verify each step before moving on.** Don't write the email module before confirming the scraper module produces the right data. - **The first run may fail.** That's not a sign of failure, that's the loop. Let Codex inspect the error output, then fix one issue at a time. The user is not technical. Walk them through reading the error and explain what's going wrong before fixing it. - **Use Codex approvals carefully.** Approve normal project actions like editing files in the project folder, installing npm packages, and running the agent. Do not approve destructive or unrelated system-level actions without explaining why they are needed. - **The user is non-technical.** Be patient. Show, don't describe. Verify their environment before assuming. If they say "it didn't work," ask them to copy-paste the exact terminal output, not summarize it. - **Don't promise this works on every utility provider.** Some portals are bot-hostile (Cloudflare challenges, CAPTCHA, MFA without app passwords). Be honest if a particular portal looks like it'll defeat this approach, and tell the user up front.