The vault already had all this context organized. 33 _context.md files tracking project state. 23 _last-session.md files with handoff notes. Signals documents tracking patterns over time. The information existed. Claude just didn't know to read it.

On the OpenClaw side (my Discord-based AI agent), this was already solved. Each Discord channel maps to a project. The agent knows which context to load based on which channel you're talking in. But Claude Code sessions are stateless. Every conversation starts fresh.

So I built a hook that fixes that.

The Problem with Instructions

The obvious first attempt: put "read the context files at session start" in CLAUDE.md. I tried that. It's guidance, not automation. Claude sees the instruction, and sometimes follows it. Sometimes doesn't. As conversations get long and context compresses, the instruction gets buried.

The second attempt: a hook that dumps all context files into every session. I have 23 session files. Most would be noise for any given conversation. "What's a good pasta recipe" doesn't need the DevX Team's last session loaded.

The real problem is intent detection. A hook fires before Claude processes your message. It can't understand what you're about to work on. It has to guess, and guessing wrong is worse than loading nothing.

Keyword Mapping

The solution is embarrassingly simple: a JSON file that maps keywords to projects.

{
  "id": "newsletter",
  "name": "Newsletter (Dev Notes)",
  "keywords": ["newsletter", "dev notes", "buttondown", "weekly newsletter"],
  "patterns": ["(write|draft|send).*?(newsletter|dev notes)"],
  "context": [
    "Publishing/_context.md",
    "Publishing/Newsletters/_context.md",
    "Publishing/Newsletters/_last-session.md"
  ],
  "suggestSkills": ["newsletter-writer", "writing-voice"],
  "priority": 2
}

When I say "let's draft tomorrow's newsletter," the hook matches "newsletter," loads the three context files, and suggests two relevant skills. No AI inference. No guessing. String matching.

The full config has 30 mappings covering every project area in the vault: Rula work projects, Modo consulting clients, publishing channels, personal projects, finances, fitness, even Cub Scouts. Each mapping defines which context files to load, which Signals doc to include, and which skills are relevant.

The Priority System

Keywords overlap. "Content" could mean the Content project or the content calendar skill. "Rula" could mean the top-level employer area or a specific sub-project like DevX.

The fix: priority levels. Higher numbers are more specific.

When "devx" matches at priority 2 and "rula" matches at priority 1, only DevX loads. The most specific match wins. When two things match at the same priority (like "content calendar" hitting both Content and Newsletter), both load. That's usually what you want.

The Hook

The hook is a bash script wired to UserPromptSubmit in Claude Code's settings. It runs every time I send a message:

1. Reads my prompt from stdin (JSON with user_prompt field)
2. Lowercases it and checks every keyword in the config
3. Falls back to regex patterns if no keyword matched
4. Filters by priority, keeping only the most specific matches
5. Outputs the file paths Claude should read

The output looks like this:

=== PROJECT CONTEXT DETECTED ===
Matched: Content / Social, Newsletter (Dev Notes)

READ these files before responding (use the Read tool):
  - Publishing/_context.md
  - Publishing/Content/_context.md
  - Publishing/Content/_last-session.md
  - Publishing/Newsletters/_context.md
  - Publishing/Newsletters/_last-session.md
  - Publishing/Stats/Signals.md (Signals)

Relevant skills for this area:
  - /content-calendar
  - /social-coach
  - /writing-voice
  - /newsletter-writer

After completing work, suggest /handoff to save session state.
=== END PROJECT CONTEXT ===

Claude sees this injected into the conversation and reads the files before responding. No match means no output. "What's the weather" produces nothing. Zero overhead on conversations that don't need project context.

The hook also writes the matched project name to a temp file. More on that in a second.

Statusline: See It at a Glance

Claude Code has a configurable statusline at the bottom of the terminal. Mine already shows the repo name, git branch, context window usage, session cost, and lines changed. I added one more piece: the active project.

The context-loader hook writes the matched project name to .claude/hooks/.current-project. The statusline script reads that file and displays it right after the repo name:

Gray Matter ⟨Newsletter⟩ git:(main) | ctx: 12% | $0.42 | +35/-8 [Opus 4.6]

No match, no file, nothing shown. When the hook fires and matches "newsletter," the statusline updates. When I switch to a general conversation, it stays on the last project (which is usually what I want, since I'm likely still in that context).

The /ctx and /handoff skills also update this file, so the statusline stays in sync regardless of how the project was set. /handoff --clear removes the file entirely, resetting the statusline for when you're done with a project and switching direction.

It's a small thing, but glancing at the statusline and seeing ⟨DevX Team⟩ confirms the hook did its job without scrolling back to check the output.

Loading Children

Some conversations span a whole area. "Let's review all publishing stuff" shouldn't load just the top-level Publishing context. It should load every sub-project underneath.

The loadChildren flag handles this. When a parent mapping has loadChildren: true, the hook runs find on the base directory and loads every _context.md and _last-session.md it finds. Publishing has six sub-projects (Blog, Content, Newsletters, Podcast, Stats, plus the parent). One keyword loads all of them.

The Write Side: /handoff

Loading context is the read side. The write side is a /handoff skill that saves session state before I close a conversation.

When I run /handoff, Claude writes a structured _last-session.md with four sections: what was done, what needs review, deferred items, and next actions. Items from the previous session that weren't addressed get carried forward automatically. The project stays active in the statusline so the next message still has context.

/handoff --clear does the same save but removes the active project from the statusline. That's for when I'm done with a project and want to switch direction mid-session. The session state is saved, the statusline resets, and the next /ctx or keyword match picks up a new project.

The file is written for a fresh agent. No "as we discussed" or references to the conversation. Just facts, file paths, and concrete next steps. Because the next agent to read it might be Claude Code on my laptop, or OpenClaw on my phone through Discord. Either way, it needs to stand on its own.

Manual Override: /ctx

The hook handles 90% of cases. For the rest, there's /ctx.

/ctx                    → list all available projects
/ctx devx               → load DevX Team context
/ctx publishing         → load all Publishing context with children

This covers three scenarios: the hook matched the wrong project, I want to switch projects mid-conversation, or I'm starting a conversation that doesn't have obvious keywords ("let's pick up where we left off" doesn't match anything useful).

Cross-System Continuity

The key design choice: the _last-session.md files are the shared state between systems. They live in the Obsidian vault. Obsidian Sync keeps them current across devices. Both Claude Code and OpenClaw read and write the same files.

The hook itself only runs locally. It's wired in .claude/settings.local.json, which is gitignored. OpenClaw never sees it because OpenClaw doesn't need it. Discord channels already provide the project routing that the hook provides for Claude Code.

The session files are the contract between systems. The routing mechanism is system-specific.

What I'd Change

The keyword list needs real-world tuning. I seeded it with obvious terms from each project's context file, but I won't know the natural language I actually use until I've run a few weeks of sessions. The first round of edits will probably happen within days.

Regex patterns are powerful but fragile. (write|draft|send).*?(newsletter|dev notes) catches "draft the newsletter" but not "I need to finish that dev notes thing." I'm keeping patterns minimal and leaning on keywords. Patterns are the fallback, not the primary matching strategy.

No learning loop yet. The system doesn't track which keywords actually fired vs. which projects I ended up working on. A log file that captures "hook matched X, user actually worked on Y" would make tuning much faster. That's probably the next thing to build.

The Stack

For anyone building something similar:

• Claude Code hooks (UserPromptSubmit) for automatic context injection
• A JSON config mapping keywords to file paths (no code changes needed to add projects)
• Claude Code skills for /ctx (manual load) and /handoff (session save)
• Claude Code statusline to show the active project at a glance
• .claude/settings.local.json to keep the hook local-only
• Obsidian Sync to share session files across devices and systems

The hook script is about 130 lines of bash. The JSON config is where all the project knowledge lives. Adding a new project means adding a JSON object with keywords and file paths. No code changes.

How It Fits Together

This is the missing piece between sessions. The vault already had project context. OpenClaw already had cross-session continuity through Discord channels. Claude Code was the gap: powerful for deep work, but amnesiac between conversations.

Now the flow works end-to-end. Start a Claude Code session, mention the project, context loads automatically. Do the work. Run /handoff. Close the session. Open Discord on my phone, pick up in the same project channel, and OpenClaw reads the same _last-session.md that Claude Code just wrote. Switch back to Claude Code tomorrow, and the hook loads the same file again.

The context follows the work, not the tool.

If this sounds interesting to you, I'd love to chat with you about it. Find me on Bluesky or X.

For podcasts I just listen to, sometimes there's nothing. No transcript, no structured notes, no way to search what was said.

I wanted the same workflow for both: give a URL, get a note with a summary, key takeaways, quotes, and a searchable full transcript. So I built a tool. URL in, structured Obsidian note out.

Here's how it works.

Step 1: Check for an Existing Transcript First

Before downloading any audio, the skill checks if the episode page already has a transcript.

Transistor-hosted shows publish transcripts at /[episode]/transcript. A quick HTML fetch finds the link. If it's there, scrape it, skip the audio entirely. Free and instant.

Other patterns to check: embedded transcript blocks in the page HTML, <podcast:transcript> tags in the RSS feed. More shows are publishing these than you'd expect.

Only if none of that works does the skill fall back to downloading audio and running it through Whisper, OpenAI's speech-to-text API.

Step 2: Get the Audio URL

If there's no transcript, I need the audio file.

The cleanest source is the RSS feed. Apple Podcasts pages embed the feedUrl in the page's JSON data. Fetch the RSS, find the right episode's <enclosure> tag, and you have a direct MP3 link.

For the first show I tested, the RSS had every episode URL, season numbers, and GUIDs for state tracking. No yt-dlp needed.

yt-dlp is a command-line tool that can download audio from just about any podcast or video URL. It's the fallback for pages that don't expose RSS cleanly.

Step 3: Transcribe with OpenAI Whisper

The Whisper API has a 25MB file limit. A typical one-hour podcast episode is 50-70MB.

The fix: split the audio with ffmpeg, a command-line tool for processing audio and video files.

chunk_duration = 1200  # 20 minutes

For a 60-minute episode, that's 3 chunks. Each goes to the Whisper API separately. The transcripts get joined. Total cost for an hour of audio: about $0.36.

The script handles the full flow: download the audio, detect file size, split if needed, transcribe each chunk, stitch the output.

Step 4: Process with Claude

The raw Whisper transcript is a wall of text. No speaker labels, no paragraph breaks, no structure.

After transcription, Claude reads it and generates:

• Summary: 2-3 sentences on what the episode covered
• Key takeaways: 5-8 bullet points with the main insights
• Notable quotes: 3-5 exact quotes pulled from the transcript
• Action items: anything actionable for the listener

Here's what a processed note looks like in practice:

## Summary
The hosts break down how their team adopted trunk-based development
after years of long-lived feature branches, and what broke along the way...

## Key Takeaways
- Short-lived branches forced smaller PRs, which made reviews faster
- Feature flags replaced branch-based isolation for incomplete work
- The hardest part was trust, not tooling
- ...

## Notable Quotes
> "We didn't have a branching problem. We had a confidence problem.
> Nobody trusted main."

The full transcript goes in the note too. I can re-run Claude on the same text later with different prompts. Last week I asked it to pull every question the host asked across five episodes. No re-transcription, no extra Whisper cost. The raw text is the asset.

Step 5: Save the Note

Each episode becomes a structured Obsidian note with frontmatter including show, season, episode, date, and source URL. The show folder gets an _index.md that uses a Dataview query to auto-populate the episode list from frontmatter. No manual maintenance as new episodes come in.

Personal/Podcasts/
  Show Name/
    _index.md              ← Dataview query, auto-populates
    2026-03-17 - Episode Title Here.md
    2026-03-10 - Another Episode Title.md
    ...

Step 6: Auto-Sync New Episodes

For shows I want to follow week over week, I added a tracking config:

// Personal/Podcasts/_tracked.json
[
  {
    "show": "Show Name",
    "rss": "https://rss.example.com/feed.xml",
    "description": "One-line description of the show.",
    "sync_from": "2026-03-18"
  }
]

A Python script reads this file, checks each show's RSS for episodes it hasn't seen (tracked by GUID), and transcribes any new ones. The sync_from date tells it where to start. Without it, a new show would backfill its entire back catalog on the first run. Per-show state lives in _sync_state.json so shows are fully independent.

A cron job runs daily at 10am ET. If any tracked show has a new episode, it gets transcribed and added to the vault automatically.

To track a new show: add an entry to _tracked.json. That's it. The cron picks it up without any other changes.

If you're building something similar: use underscore prefixes (_sync_state.json), not dotfiles. Obsidian Sync skips dotfiles by default, which means your state file won't sync across devices. Learned that the hard way.

What It Actually Cost to Build

A few hours. The transcription script is about 160 lines of Python. The sync script is about 300. The Obsidian skill that ties it together is a markdown file.

Tools used:

• OpenAI Whisper API for transcription (~$0.006/min)
• ffmpeg for audio splitting and duration detection
• yt-dlp as a fallback for audio extraction
• Python 3 for glue code (stdlib only, no pip packages)
• Claude for post-processing the raw transcript into structured notes

The whole thing runs on the same DigitalOcean droplet that runs OpenClaw. No new infrastructure.

What I'd Add Next

Speaker diarization is the obvious gap. Whisper doesn't identify who's speaking. AssemblyAI and Deepgram both have this built in for similar pricing. For any show with consistent hosts and guests, labeled transcripts would make the notes far more useful.

I'll swap the transcription backend when I have a reason to re-transcribe.

The other thing: cross-episode analysis. Once I have a full season of transcripts, I can ask Claude to find every time the hosts discussed a specific topic across all episodes. That's a skill worth building once there's enough content in the vault.

A month ago, I couldn't search anything from the podcasts I listen to. Now every episode lands in the vault with structure, summaries, and a full transcript I can query whenever I want. The vault gets richer without extra work. Some podcasts publish transcripts. Some don't. I stopped thinking about which is which.

If you're building something similar I'd love to chat with you about it. Find me on Bluesky or X.

Sunday night at 9 PM, a scheduled task kicks off on my OpenClaw server. It syncs my Bluesky posts, Twitter activity, newsletter stats, and podcast downloads into the vault. Then it aggregates everything into a cross-platform review. Then it generates next week's content calendar with seven days of social posts, newsletter angles, podcast topics, and blog candidates. All from the same vault the agent reads and writes to every day.

I didn't plan to build a content flywheel. I built individual tools to solve individual problems, and they connected into a loop.

This is how the pieces fit together, what the weekly cycle actually looks like, and how the system improves its own writing over time.

The Sunday Night Pipeline

The whole thing runs as a single chained command in OpenClaw:

/bluesky-sync → /twitter-sync → /buttondown-sync → /transistor-sync → /social-review → /content-calendar

Each step is a Claude Code skill. Each one reads from the vault and writes back to it. The four syncs run as separate scheduled jobs. Once they've finished, the review runs on their combined output, then the calendar.

Here's what each step does:

Platform syncs pull raw data into markdown files. /bluesky-sync creates Publishing/Stats/Bluesky/Bluesky 2026-W12.md with every post, reply, like count, and follower change for the week. Same pattern for Twitter, Buttondown (newsletter), and Transistor (podcast). Each file has structured frontmatter for the numbers and a content section with the actual posts.

/social-review reads all four sync files and produces a combined stats review. It compares engagement rates across platforms, identifies which posts performed best, tracks what content types are working (original observations vs. promo vs. personal), and maintains a running Signals document. Signals are patterns that persist across weeks: "Twitter outperforming Bluesky on cross-posted content" or "original content outperforms promotional." When a signal shows up three weeks in a row, it gets promoted to a strategy change.

/content-calendar reads the stats review, the signals, work logs, recent newsletters, podcast episodes, and a social backlog of unused post ideas. It produces a daily content calendar for the upcoming week: one post per day, Monday through Sunday. Each post includes the draft text, content type, source references, sanitization flags, and empty Posted: blocks with Twitter and Bluesky link placeholders for me to fill in after posting.

The calendar also suggests newsletter opening thought angles, podcast topics, and blog candidates for the week. All sourced from what's already in the vault.

The Backlog System

Not every idea fits into this week's calendar. Some are too early, some need more distance from work, some are waiting for a triggering event. These used to disappear. Now they go to backlogs.

Four backlog files, one per medium:

Each backlog has three sections: Active (ready to use), Potential (parked, not timely yet), and Ignored (stop suggesting).

The /content-idea skill captures ideas mid-week and routes them to the right backlog. If an idea spans multiple mediums, it writes to each one and cross-links them with Related: fields. A social post about "AI context switching overhead" that could also be a newsletter angle gets entries in both backlogs, each pointing to the other.

When the content calendar generates next week, it reads the social backlog's Active section and can pull ideas directly into the schedule. Unposted calendar items flow back to the backlog. Items that sit in Active for two weeks without being posted get moved to Potential during grooming.

The backlogs are the memory between weeks. The calendar is this week's plan.

The Voice Loop

The part I didn't expect to build: the system that teaches itself how I write.

The content calendar drafts social posts in my voice using a /writing-voice skill. That skill is a detailed reference document. It covers how I structure arguments, my default sentence length, how humor shifts between social and newsletters, emoji habits, hashtag patterns. Specific enough that AI-drafted posts are close to what I'd write.

Close, but not right. I still rewrite most posts before publishing.

The content calendar captures both versions. The AI draft sits in the blockquote under each day. My rewritten version goes in the Posted: block. Over time, these pairs accumulate.

Once a month, I run /voice-review. It reads every AI draft and Posted pair across the past 4+ weeks. It categorizes the edits I made: tone shifts, length changes, added specifics, dropped sarcasm, added emoji. It tallies patterns across weeks and proposes updates to three skills:

1. writing-voice (the reference document itself)
2. social-coach (the skill that reviews posts for voice accuracy)
3. content-calendar (the skill that drafts the posts)

The proposals only apply if a pattern shows up in 3+ posts or across 2+ separate weeks. One-off edits are noise. Consistent rewrites are signal.

After the first review, I found that the AI was consistently writing resigned, self-deprecating social posts ("classic.", "I don't know, maybe write the doc.") and I was consistently rewriting them toward genuine enthusiasm ("Super excited!", "Docs save time!"). The writing-voice skill now explicitly says: dry humor belongs in newsletters and podcasts, social posts get sincere energy. The next round of drafts should be closer.

The flywheel: write → post → review → update the voice → write better next time.

The Full Weekly Cycle

Here's how a typical week runs end to end:

Sunday night (automated): Platform syncs → social review → content calendar generated. I wake up Monday with a plan.

Monday through Sunday: I review each day's post, edit if needed, post to Twitter and Bluesky, fill in the Posted: block and links. Mid-week ideas get captured with /content-idea and routed to the right backlog.

Thursday: I run /newsletter-writer to draft it, pulling from the vault's work logs, recent content, and newsletter backlog.

Friday: Newsletter goes out.

Saturday: Podcast publishes. I run /import-podcast to pull the episode into the vault, then /podcast-review to update the topics backlog.

End of month: /voice-review compares AI drafts to actual posts and proposes skill updates. /review-monthly aggregates weekly reviews and forces promote-or-drop decisions on lingering signals.

Every step reads from and writes to the same vault. The output of one skill becomes the input of another. The stats review feeds the content calendar. The content calendar produces drafts. The drafts get rewritten. The rewrites teach the voice skill. The voice skill improves the next round of drafts.

What I'd Change

The 70/20/10 mix target needs rethinking for daily posting. The original content strategy was built for 3-4 posts a week. Seven posts a week shifts the math. Some weeks are heavier on personal content because there are only so many value observations per week. I'm still calibrating.

Backlog grooming needs a trigger, not just a timer. The "2 weeks in Active then move to Potential" rule is fine for social posts. For blog and newsletter ideas, timeliness varies more. A blog idea about a conference talk might sit for months and still be valid. I'll probably split the aging rules by medium.

The voice review needs more data. One month of before/after pairs showed clear patterns. But the sample is small. The real test is whether the second month's drafts need fewer rewrites. I'll know in about a month.

The Stack

For anyone who wants to replicate this:

• Obsidian as the knowledge base (any vault structure works, but context files and signals make it much better)
• Claude Code for building skills (each skill is a markdown file with instructions, no traditional code)
• OpenClaw on a DigitalOcean droplet ($24/month) for the always-on agent
• Obsidian Headless to keep the server and your Mac in sync
• Discord as the chat interface (each channel maps to a project)
• Bluesky, Twitter/X, Buttondown, Transistor as publishing platforms (the sync skills pull from their APIs)

The total stack cost is about $29/month: $24 for the server and $5 for Obsidian Sync. I also bought $20 in Twitter API credits to pull weekly stats, but that should last a while at this usage level. Everything else is free or already paid for.

26 Claude Code skills power the whole system at time of writing. Each one is a markdown file. No Python scripts, no cron jobs, no custom API integrations. The skills are instructions the AI follows. If the workflow changes, I edit the markdown.

How It Started

I didn't set out to build a content system. I set out to stop forgetting things.

The work logs started because I needed a brag doc for performance reviews. The stats syncs started because I wanted to know which platform was worth my time. The content calendar started because I was posting randomly and wanted a plan. The voice review started because the AI drafts were too polished and self-deprecating, and I kept rewriting them toward something more direct.

Each tool solved a specific problem. The flywheel emerged because they all read and write to the same vault.

Hit me up on Bluesky or X if you're building something similar. I'd like to hear how others are approaching this.

This week I did something about it.

I restructured my Obsidian vault into a format AI agents can navigate, built an automation layer with Claude Code skills, and deployed OpenClaw on a DigitalOcean droplet with headless Obsidian Sync tying it all together. The vault is the brain. The agent reads it, writes to it, and picks up where the last session left off.

This is how I approached it, what I learned, and what I'd change.

The Problem: Smart Tools, No Memory

Claude is fast. Claude Code is faster. But every session starts from zero. You open a new thread, re-explain the project, re-describe the architecture, re-state your preferences. The AI forgets everything between conversations.

I tried to solve this with longer system prompts, CLAUDE.md files in repos, and careful copy-pasting of context between sessions. It helped. It wasn't enough.

The real issue: I have a dozen active projects across my day job, a consulting business, a podcast, a newsletter, and personal projects. No single prompt can hold all of that. And the projects connect to each other in ways that matter. A work observation becomes a newsletter angle. A podcast conversation surfaces a blog post idea. A side project teaches a lesson that applies to the day job.

I needed a persistent, structured knowledge base that any AI session could read and write to. I already had one. I just needed to reorganize it.

Step 1: Restructure the Vault

My Obsidian vault ("Gray Matter") had been a flat collection of project folders with inconsistent formats. Some projects had detailed context files. Most didn't. Meeting notes lived in one place, work logs in another, but nothing explicitly connected them.

The restructuring introduced three conventions:

Domain-based hierarchy. Everything lives under Work/, Publishing/, or Personal/. Work splits by employer or client. Projects nest under their parent area.

Context files per project. Every project gets three files:

• ProjectName.md with a stable summary, stack, and key links
• _context.md with active state: current work, open decisions, blockers, signals, key people, and notes for the agent
• _last-session.md with what was done, what needs review, and next actions

Context stacking. At session start, an agent reads files in order: area context, project context, signals, last session. Each layer adds specificity without repeating what's above it. A session about a specific work project reads the area context, then the team context, then the project context, then signals, then the last session file. Five files, full picture.

Context files are session memory. _context.md answers "what should an AI know right now?" and _last-session.md answers "what happened last time?" Together, they eliminate the re-explanation problem.

Step 2: Build the Automation Layer

Restructuring the vault was the foundation. Making it self-maintaining was the real unlock.

I built a suite of Claude Code skills that handle the weekly workflow:

• /review-daily enriches work log entries with data from GitHub, Jira, Confluence, and meeting notes
• Platform syncs (/bluesky-sync, /twitter-sync, /buttondown-sync, /transistor-sync) pull engagement data into the vault
• /social-review aggregates stats across all platforms and maintains a running Signals document
• /review-weekly and /review-monthly generate structured reviews from the week's or month's raw data
• /content-calendar mines the vault for post ideas, checks what resonated last week, and produces a week of social posts, newsletter angles, and podcast topics
• /summarize-meetings generates frontmatter summaries for meeting notes based on their content
• /import-podcast pulls in new episodes from Transistor

Each skill reads specific vault files and writes its output back to the vault. The vault gets richer over time without manual effort. The content calendar skill reads work signals, publishing stats, recent newsletters, and podcast topics to suggest posts. It knows what performed well last week because the stats are in the vault. It knows what I'm working on because the work logs are in the vault.

The skills started as local Claude Code commands. Now they run from the server on a schedule or on demand through Discord.

Step 3: Deploy OpenClaw

OpenClaw is a self-hosted AI agent runtime. You interact with it through Discord (or WhatsApp, or other channels). It reads your files, runs commands, and maintains persistent context across sessions.

The setup:

DigitalOcean droplet (4GB RAM, 2 vCPU, $24/month). Runs Ubuntu with Node 22. OpenClaw, obsidian-headless, and pm2 for process management.

Obsidian headless sync. This is the piece that makes the architecture work. obsidian-headless is a new official Obsidian product (released February 2026) that syncs your vault to a server without the desktop app. Changes I make in Obsidian on my Mac sync to the server. Changes the agent makes on the server sync back. The vault is the shared state layer.

Mac (Obsidian) <--sync--> Server (obsidian-headless) <--reads/writes--> OpenClaw

Discord as the interface. Each Discord channel maps to a project domain. I message the bot, it reads the relevant context files from the vault, and responds with full project awareness. The sessions persist per-channel, so a conversation about a project picks up where I left off.

The channel-to-project mapping lives in openclaw.json. Each entry points to a vault path and injects a system prompt telling the agent what it's working in:

"channels": {
  "discord": {
    "guilds": {
      "YOUR_GUILD_ID": {
        "requireMention": false,
        "users": ["YOUR_USER_ID"],
        "channels": {
          "CHANNEL_ID_WORK_PROJECT": {
            "systemPrompt": "This is the #work-project channel. Project vault path: /vault/Work/Company/Project/. On first message, load context following the order defined in Claude.md: area _context.md → project _context.md → Signals (if present) → _last-session.md → related projects. After completing any meaningful task, decision, or topic, update _last-session.md (frontmatter + What we did / Changes made / Open work). If project state changed, also update _context.md — overwrite stale content, do not append."
          }
        }
      }
    }
  }
}

Every channel follows the same pattern: vault path + context stacking instructions. The agent knows where it is, what to read first, and what to write when the session ends. Adding a new project is one new entry in this config.

Cloudflare Tunnel for the Control UI. OpenClaw ships a browser-based Control UI for managing sessions, config, and devices. The gateway binds to loopback by default, so nothing is publicly exposed. I use a Cloudflare Tunnel to access it remotely without opening any ports, with a Cloudflare Access PIN policy as an outer auth gate. The OpenClaw token and device pairing are the inner gates. Three layers, no open ports.

Skills on the server. Obsidian Sync doesn't sync hidden directories, so .claude/skills/ needs its own path. I set up a separate git repo for skills. Both my local machine and the server pull from it. OpenClaw discovers skills automatically from the workspace directory.

The whole deployment took an afternoon. Most of the time was configuring Discord bot permissions and testing the sync pipeline end-to-end.

What This Actually Looks Like Day to Day

Monday morning. I open Discord on my phone and ask the publishing channel: "What's on the content calendar this week?" OpenClaw reads Publishing/Content/2026-W12.md and gives me the rundown. I tell it to adjust Wednesday's post. It updates the file. The change syncs to my Mac by the time I sit down.

During a meeting, I notice a pattern worth tracking. After the meeting, I tell the work channel: "Add a signal about review bottlenecks increasing since the AI rollout." It appends to Work/Signals.md.

Friday evening, I run the weekly review. The agent reads five days of work logs, meeting notes, and signals, then generates a structured review with metrics, recurring themes, and brag doc candidates. It writes the review to the vault. I read it in Obsidian on my couch.

The agent isn't doing anything I couldn't do manually. It's doing the things I wouldn't get around to. The weekly reviews, the stats syncs, the signal tracking. The maintenance work that makes the vault useful but never feels urgent enough to do by hand.

The Architecture Decision That Matters Most

The vault is the single source of truth. Not OpenClaw's memory. Not a database. Not a separate knowledge graph. Plain markdown files in Obsidian, synced everywhere.

Portability. If OpenClaw disappears tomorrow, the vault is still there. Every context file, every review, every signal. The data isn't locked into an agent framework. It's markdown.

Readability. I can open any file in Obsidian and see exactly what the agent knows. No black-box memory systems. No embeddings I can't inspect. If the agent has wrong context, I edit a markdown file.

Composability. Claude Code reads these files locally. OpenClaw reads them on the server. A future agent I haven't built yet could read them too. The format is the interface.

What I'd Do Differently

Start with fewer context files. I populated context files for every project at once. It was a lot. Better to start with your 2-3 most active projects and expand as you go.

Test the sync pipeline earlier. I spent time getting the vault structure right before deploying obsidian-headless. Should have set up the sync first and iterated on the structure with the agent already running. Seeing how the agent actually uses the files changes how you write them.

Keep _context.md ruthlessly current. A context file that's two weeks old is worse than no context file. It gives the agent confident but wrong information. The fix: build the update instructions directly into the channel system prompt. Instead of "update at the end of a session" (which never fires on Discord, since sessions don't have a clear end), use "after completing any meaningful task, update both _last-session.md and _context.md." Give the agent an explicit format for each file. The agent writes mid-conversation, not on some hypothetical session close that never happens.

Try It Yourself

You don't need OpenClaw to start. The vault structure works with Claude Code today.

1. Pick your top 3 projects. Create _context.md and _last-session.md for each one. Write them for an AI reader, not for yourself. What does a new session need to know?

2. Add a CLAUDE.md at the vault root. Tell the agent where things live, what the conventions are, and what to read first. This is your vault's instruction manual.

3. Add a USER.md. Who are you, what do you work on, and what does the agent need to know to be useful? Keep it under 300 words.

4. Update _last-session.md after meaningful work. This is the handoff note to your next session. What did you do, what's pending, what's next. If you're using OpenClaw with Discord channels, build this into the channel system prompt so the agent writes after each task automatically. Don't rely on "end of session" as a trigger; it never fires.

5. Build one automation skill. Start with something you do every week that pulls data from multiple sources. The weekly review is a good first candidate.

6. When you're ready for always-on, deploy OpenClaw. A $24/month droplet, obsidian-headless for sync, Discord for the interface. Map each Discord channel to a project in openclaw.json. The vault you already structured is the brain.

Your AI gets better when it has memory. The simplest memory system is files it can read.

If you try this approach or have a different solution to the multi-session problem, find me on Bluesky or X.

My site had no space for my newsletter archive, my podcast, my projects, or my talks. It was just a blog with a custom domain. My content was scattered across Hashnode, Buttondown, Transistor, and a few other places with no unified home. I wanted a single hub for everything I publish, built with the tools I know: Laravel, Filament, Blade, Tailwind. So I rebuilt chrisgmyr.dev from scratch as a self-hosted Laravel application.

The whole thing, from first conversation to deployed on Laravel Cloud, took about 24 hours (not working the whole time, of course). I used Claude AI for design and Claude Code for implementation. This is how I approached it.

Step 1: Design First, In Chat

Before touching code, I spent time in the Claude chat interface talking through the design. Most people skip this step. It's the most valuable one.

I didn't show up with a spec. I showed up with a handful of personal sites from other developers I respect, plus a vibe: dark-first, clean but distinctive, personality through typography and subtle interactions.

Before Claude made any recommendations, it ran a research pass, reviewing 14 developer blogs, dark mode design systems, typography pairings, and multi-content architecture patterns. That's not something I asked for explicitly. It's just what happened when I treated the conversation as a design session rather than a prompt. From that research, we worked through specifics together.

Colors. We landed on amber (#F59E0B) as the accent instead of the blue or green you see on every developer site. Amber is warm, pairs well with dark backgrounds, and clears accessibility contrast requirements. We mapped out a full palette for dark and light modes.

Typography. Space Grotesk for headings, Inter for body text, JetBrains Mono for code and metadata. Using a monospace font for dates, tags, and reading times gives the site a subtle technical feel without needing custom illustrations.

Layout. A bento grid homepage that shows all content types at once: blog posts, latest podcast episode, newsletter signup, featured project, and recent talks, all in one responsive grid. Plus /now and /uses pages for the personal context stuff I always end up wanting to reference.

Interactions. A cursor-following amber glow on the background, a reading progress bar on blog posts, cards that scale on hover with border color shifts, and animated link underlines. Small details that make the site feel alive without being distracting.

The key was treating Claude as a design collaborator, not a search engine. I pushed back on suggestions. I asked "why amber over teal?" and refined based on the reasoning. The conversation naturally produced a complete design system.

Step 2: Write the Plan Document

After the design conversations, I compiled everything into a single markdown file: chrisgmyr-dev-project-plan.md. Think of it as a detailed project plan and design spec combined.

The document covered:

• Design system: every color value, typography choice, spacing rule, and interactive detail
• Site architecture: routes, page layouts with ASCII wireframes, navigation structure
• Data models: migration schemas for posts, tags, newsletters, episodes, projects, talks, and pages
• Filament admin setup: resources to create, Block Builder blocks, dashboard widgets, and Unsplash integration (a dedicated UnsplashService, a custom picker modal inside Filament, attribution rendering on the frontend, and responsive image delivery via Unsplash CDN URL params)
• File structure: every controller, model, service, and view mapped out
• Implementation phases: six phases with specific tasks and acceptance criteria
• Package recommendations: what to install and why

The document ran about 800 lines of markdown. That might sound like overkill. It saved days of back-and-forth during the build.

When Claude Code had a question about how something should work, the answer was already in the plan. The quality of AI output is directly tied to the quality of your input. A vague "build me a blog" gets you a generic blog. A plan with wireframes, color values, and acceptance criteria gets you something that matches your vision.

Step 3: Build Phase by Phase with Claude Code

With the plan in place, I opened Claude Code and worked through the phases one at a time. Each phase had a clear scope and acceptance criteria, so I knew when to move on.

Phase 1: Foundation

Scaffolding: Laravel 12, Filament 5, Tailwind CSS 4, all migrations and models, the layout with nav and footer, and the theme toggle. Claude Code generated the initial structure based on the plan. I reviewed every file before committing.

Phase 2: Filament Admin

Seven Filament resources (Posts, Tags, Newsletters, Episodes, Projects, Talks, Pages), the Block Builder, an Unsplash picker modal for hero images, and dashboard widgets. This phase was the most Filament-heavy. Having the correct namespaces and patterns in the plan kept things on track.

Phase 3: Public Pages

Controllers and Blade views for every page. Blog rendering with markdown-to-HTML conversion, syntax highlighting with Prism.js, newsletter signup forms that post to Buttondown, and RSS feed generation.

Phase 4: Homepage Bento Grid

The responsive CSS Grid homepage pulling real data from all content types. This was the page I was most excited about. Seeing it come together with live data was satisfying.

Phase 5: Visual Polish

The cursor-following glow effect, reading progress bar, scroll-triggered animations, Open Graph meta tags, a custom 404 page, and the sitemap. The details that tell visitors you cared.

Phase 6: Content Migration

Importing blog posts from Hashnode, newsletter archive from Buttondown, podcast episodes from Transistor's API, and setting up 301 redirects so old URLs kept working.

The Simplify Pass

Here's a workflow I want to highlight separately, because it changed how I think about building with AI.

After Claude Code finished each phase, I'd review the output, then ask it to simplify. Look at the git history:

d1bbcb4 Phase 1: Foundation - models, migrations, layout, design system, routes
198d00e Phase 2: Filament admin - all content resources, dashboard, Unsplash picker
d2ed096 Simplify Phase 2: extract SlugHelper, fix duplicate labels, remove dead code
b8c72fa Phase 3: Wire up markdown rendering, prose styles, syntax highlighting
aba807a Simplify Phase 3: remove duplicate CSS, fix Prism timing, clean deps
50f9ec7 Phase 4: Homepage bento grid with all content types
8fa343d Simplify Phase 4: fix YouTube ID extraction, replace inline JS hover
ae9d3ab Phase 5: Visual polish, OG images, 404 page, sitemap
4b6e2a4 Simplify Phase 5: cache sitemap, add Project screenshot accessor
b052095 Phase 6: Content migration tooling and Hashnode redirects
331c4c9 Simplify Phase 6: fix cover images, canonical URLs, route structure

The first pass gets the functionality right. The second pass cleans it up: removes dead code, extracts shared helpers, fixes edge cases, tightens the CSS. This two-pass approach works well with AI code generation. Let it build fast, then review and refine.

Building Away from My Desk with Claude Code Remote Sessions

One feature I used a lot that I haven't seen written about much: Claude Code remote sessions.

The workflow: start a Claude Code session locally on my Mac, then connect to that same session from the Claude mobile app. This let me review progress, answer Claude's questions, and steer the implementation while away from my desk, during a lunch break, sitting on the couch after the kids went to bed, or waiting somewhere.

Claude Code will pause and ask for input when it hits a decision point or needs clarification. Without remote sessions, those pauses meant coming back to a stalled terminal. With remote sessions, I could respond from my phone, keep the session moving, and come back to a desk with more work done.

If you're running longer Claude Code sessions on a personal project, set up remote access. The async nature of the workflow fits evening-and-weekend building well.

What Worked

The plan document was the most important investment. Claude Code referenced it for architectural decisions, design values, and implementation details. No ambiguity. If you take one thing from this post, spend time on the plan before touching code.

Phase-based builds kept scope manageable. Each phase had clear acceptance criteria. I knew when something was "done enough" to move on.

The simplify pass caught real problems. Duplicate CSS selectors, dead code from refactoring, a timing bug with Prism.js initialization, inline JavaScript that should have been Alpine.js directives. A second review pass always finds something worth fixing.

Reviewing every commit mattered. I didn't rubber-stamp the AI output. I pushed back on approaches I didn't like and made manual tweaks where my taste differed from what was generated. The AI wrote the bulk of the code. I steered the architecture and design decisions.

What I'd Do Differently

Tests earlier. I added the test suite after most of the building was done. Writing tests alongside each phase would have caught bugs sooner.

Smaller phases. Phase 2 (Filament admin) and Phase 3 (public pages) were both large. Smaller chunks would have made the simplify passes more focused.

Screenshot-driven iteration. For visual polish, I should have captured screenshots after each change to track the progression. It would make this post more visual too.

Try It Yourself

If you want to take a similar approach on your own project:

1. Chat first, code later. Spend time in Claude AI talking through your design, architecture, and content strategy. Push back. Ask why. Refine until you have a clear vision.

2. Write the plan. Compile the conversation into a structured document with color values, wireframes, data models, file structure, and phased tasks. Be specific. "Dark mode" isn't a plan. "#0E0E10 background, #18181B surface, #F59E0B accent" is.

3. Phase your build. Break the work into chunks with acceptance criteria. Two to four days per phase. Commit after each one.

4. Simplify after each phase. Review the output, then run a cleanup pass. Remove dead code, extract shared logic, fix edge cases.

5. Use remote sessions. If you're building in short windows of time, connect your local Claude Code session to the mobile app. You can keep things moving without being at your desk.

6. Own the decisions. The AI writes the code. You own the taste, the architecture, and the tradeoffs. If something doesn't feel right, change it.

The site is live at chrisgmyr.dev. It's fast (server-rendered Blade, minimal JavaScript), looks distinctive, and gives me a single home for everything I publish.

Claude AI for design conversations and Claude Code for implementation felt like having a design partner and a senior engineer available at odd hours. The plan document was the bridge between the two. Writing it was the best time investment of the whole project.

If you try this approach, I'd love to hear about it. Find me on Bluesky or X.

I used to use a fantastic Chrome extension called Sprout for entering transactions, but unfortunately the developer took the extension off the Chrome store and stopped development. I wanted something that could be equally as fast and easy to use.

Over this blog series, we'll utilize the PHP Laravel framework and its built-in tools to build a robust CLI application and interface with YNAB's API.

The CLI functionality will not replace the excellent web UI but only introduce faster ways to alleviate everyday tasks to go about our day. It's a perfect fit since I'm usually in my terminal during the day.

Before we begin, if you aren't a YNAB user, please sign up. Using my referral link will give you a 34 day trial and if you decide to purchase a subscription, YNAB will give us both a free month!

Set up Laravel

To get started, install Laravel from the docs. I usually use the Laravel installer, but do whatever works best for you. At the time of writing, this will install a new Laravel 11 application.

laravel new ynab
cd ynab

Because we want an easy to use interface within the CLI, we'll also want to install Laravel Prompts.

composer require laravel/prompts

Next, we'll want to add config values for our YNAB API key and the default budget ID that we'll want to utilize. Right now, I'm only setting this up as a single budget, but in the future we could make this more configurable, or you can do this on your own in your own application.

Within .env and .env.example, add

YNAB_TOKEN=
YNAB_BUDGET_ID=

Within config/services.php, add

    'ynab' => [
        'token' => env('YNAB_TOKEN'),
        'budget_id' => env('YNAB_BUDGET_ID'),
    ],

To get a YNAB API key, go into your Account Settings, scroll toward the bottom and within the Developer Settings click the Developer Settings button. On the next page, under Personal Access Tokens, click the New Token button. This will reveal your API token that you can paste into your .env file.

The budget id is the UUID in the URL when viewing your budget in the YNAB web UI, so you can copy/paste if from there, or get it from the API (/budgets).

Create Transaction Command

Next, we'll want to set up our initial artisan command, so we'll run php artisan make:command CreateTransaction, which will create app/Console/Commands/CreateTransaction.php. Then, we'll stub out the initial command class.

class CreateTransaction extends Command
{
    protected $signature = 'ynab:transaction';
    protected $description = 'Creates a new transaction in YNAB';

    public function handle(): int
    {
        return self::SUCCESS;
    }

Before we get too far, let's set up a shared trait for the HTTP client to be shared across all commands, since we'll be creating more commands in the next blog posts. Create app/Console/Commands/Concerns/Ynab.php with the following code

<?php
declare(strict_types=1);

namespace App\Console\Commands\Concerns;

use Illuminate\Http\Client\PendingRequest;
use Illuminate\Support\Facades\Http;

trait Ynab
{
    protected function getClient(): PendingRequest
    {
        return Http::baseUrl('https://api.ynab.com/v1/')
            ->acceptJson()
            ->asJson()
            ->throw()
            ->withToken(config('services.ynab.token'));
    }
}

This helper method will allow us to remove the boilerplate of setting up the YNAB client and set up JSON handling, throw an exception if there's an API error, and use the token from the config file for authentication.

Then, we'll include that in the transaction command class

class CreateTransaction extends Command
{
+    use Concerns\Ynab;
+
    protected $signature = 'ynab:transaction';
    protected $description = 'Creates a new transaction in YNAB';

Similar to the Sprout UI, we'll want to build a CLI to handle:

• Amount

• Account

• Payee

• Category

• Optional memo

• Optional flag

• Optional cleared/uncleared

For now, we'll skip date (we'll assume today's date), approval (default to true), split transactions, and other options found in the Sprout and YNAB UIs. For reference, here is the Sprout UI.

Gathering Budget Data

Before we start building our CLI UI, we'll need to get initial budget data from YNAB. You can find the API docs here. For now, we'll want to gather

• Accounts

• Payees

• Categories

So, we'll stub those out in our command

class CreateTransaction extends Command
{
    use Concerns\Ynab;

    protected $signature = 'ynab:transaction';
    protected $description = 'Creates a new transaction in YNAB';

    public function handle(): int
    {
+        $accounts = $this->getAccounts();
+        $payees = $this->getPayees();
+        $categories = $this->getCategories();
+
        return self::SUCCESS;
    }

Getting Budget Accounts

    public function getAccounts(): Collection
    {
        $response = $this->getClient()->get('budgets/' . config('services.ynab.budget_id') . '/accounts');

        return collect($response->json('data.accounts'))
            ->filter(fn($account) => $account['on_budget'] && !$account['closed'])
            ->mapWithKeys(fn($account) => [$account['id'] => $account['name']]);
    }

In this step, we're

1. Getting the raw response from the /accounts endpoint from the API

2. Grabbing the data from the response under data.accounts

3. Filtering for only accounts "on budget" and not closed. On budget means that the money is used for specific budgeting purposes. Alternatively, there are "tracking" accounts that don't impact your budget positively or negatively.

4. Creating a new collection of account IDs matching their name

Getting Budget Payees

    public function getPayees(): Collection
    {
        $response = $this->getClient()->get('budgets/' . config('services.ynab.budget_id') . '/payees');

        return collect($response->json('data.payees'))
            ->reject(fn($payee) => $payee['deleted'])
            ->mapWithKeys(fn($payee) => [$payee['id'] => $payee['name']]);
    }

In this step, we're

1. Getting the raw response from the /payees endpoint from the API

2. Grabbing the data from the response under data.payees

3. Removing any deleted payees

4. Creating a new collection of payee IDs matching their name

Getting Budget Categories

This step is a little more involved because the /categories endpoint has top-level Category Groups, then includes the categories data within each group. You can see the schema for CategoryGroupWithCategories here.

    public function getCategories(): Collection
    {
        $response = $this->getClient()->get('budgets/' . config('services.ynab.budget_id') . '/categories');

        return collect($response->json('data.category_groups'))
            ->reject(fn($categoryGroup) => $categoryGroup['hidden'] || $categoryGroup['deleted'])
            ->pluck('categories')
            ->flatten(1)
            ->reject(fn($category) => $category['hidden'] || $category['deleted'] || $category['category_group_name'] === 'Credit Card Payments' || $category['category_group_name'] === 'Internal Master Category')
            ->mapWithKeys(fn($category) => [$category['id'] => $category['category_group_name'].': '.$category['name']]);
    }

In this step, we're

1. Getting the raw response from the /categories endpoint from the API

2. Grabbing the data from the response under data.category_groups

3. Removing hidden and deleted category groups

4. Grabbing the categories array from each group

5. Flattening the collection so all of the categories are in the same root of the collection

6. Removing any categories that are hidden, deleted, or match specific YNAB-generate categories we don't need

7. Creating a new collection of category IDs matching the category group name with the category name

Whew, we did it! Now, we'll move on to the Laravel Prompts UI.

Leveraging Laravel Prompts for the CLI UI

Prompts has numerous options for CLI inputs. These can either be one-off or chained together into a form. A form enables us to have multiple questions and inputs while collecting the responses to be used later. Since we'll have a number of questions to ask, we'll be using a form.

        $responses = form()
            ->text('Amount', required: true, name: 'amount')
            ->select('Account', options: $accounts, required: true, name: 'account')
            ->search(
                label: 'Payee',
                options: fn (string $value) => strlen($value) > 0
                    ? $payees->filter(fn ($payee) => Str::contains($payee, $value, true))->all()
                    : [],
                name: 'payee'
            )
            ->search(
                label: 'Category',
                options: fn (string $value) => strlen($value) > 0
                    ? $categories->filter(fn ($category) => Str::contains($category, $value, true))->all()
                    : [],
                name: 'category'
            )
            ->text('Memo (optional)', name: 'memo')
            ->select('Flag color (optional)', options: ['none', 'red', 'orange', 'yellow', 'green', 'blue', 'purple'], name: 'flag_color')
            ->select('Cleared', options: ['cleared', 'uncleared'], default: 'uncleared', name: 'cleared')
            ->submit();

There's a lot to unpack here, but it should be straight forward

1. We'll ask for the amount of the transaction - either positive (income) or negative (expense)

2. Select the account from a select prompt loaded with our accounts from the API

3. A search box for the payee loaded from the API. As new characters are added, the search suggest auto-updates and allows you to arrow up/down to select the payee. The options function searches and returns the values the match the input given.

4. Similar to the payees, the categories work the same way with a search suggest

5. Memo is a simple text field and also optional

6. Flag color is also optional using a select, like accounts. YNAB allows you to use their default flags, or change them in the UI. I only use these occasionally.

7. Cleared or uncleared transaction using another select. If I already know the transaction has cleared the bank/card, then I'll change to cleared but otherwise most transactions are new and haven't hit the account yet, so we'll default to uncleared

8. Finally, submit the responses

Creating the new transaction via the API

The Laravel Prompts form will return an array of $responses that will contain the key/value from what's provided in each input's name. We'll use the /transactions endpoint to combine our data and use some sensible defaults for other transaction-related data points.

        $response = $this->getClient()->post('budgets/' . config('services.ynab.budget_id') . '/transactions', [
            'transaction' => [
                'account_id' => $responses['account'],
                'payee_id' => $responses['payee'],
                'category_id' => $responses['category'],
                'amount' => $responses['amount'] * 1000,
                'memo' => $responses['memo'],
                'flag_color' => $responses['flag_color'] === 'none' ? null : $responses['flag_color'],
                'cleared' => $responses['cleared'],
                'date' => now()->toDateString(),
                'approved' => true,
            ],
        ]);

1. The account will be the account's uuid from YNAB

2. The payee will be the payee's uuid

3. The category will be the category's uuid

4. The amount (positive or negative) needs to be converted to what YNAB calls milliunits or 3 decimal places, so we'll multiply our amount by 1,000 (docs)

5. Optional memo text

6. Optional flag color, or converting none to null in case a color isn't specified

7. Cleared value

8. Current date - I usually add transactions as they happen, so we'll default to the current date. We might make this configurable in the future.

9. Approved defaulted to true - YNAB will highlight any new transactions via their auto-import process which you can either approve or delete. However, for the CLI's purposes, I'm entering the transaction manually, I'm approving at the same time.

Handling API responses

At this point, I'm not too worried about responses and everything has been working smoothly so far. We'll expand on this in the future, but again, we'll keep things simple right now.

        if ($response->successful()) {
            $this->info('Expense created successfully');
        } else {
            $this->error('Failed to create expense - ' . $response->json('error.detail'));
        }

        return self::SUCCESS;

Wrapping Up

I hope you enjoyed this quick journey through setting up a fresh, new, Laravel application with Prompts and the YNAB API. As stated earlier, I'll be adding more blog posts to this series as I work through other command and refactorings along the way. I'm sure you can already see some areas that could use improvement. Further, the YNAB API offers some interesting options like Delta Requests, Rate Limiting, and better error handling that we can explore later as well.

Live Demo

Check out the repo on GitHub

• laravel-ynab-cli

• PR #1 for the CreateTransaction command

Listen to the podcast episode

Subscribe

Please subscribe to my newsletter, RSS feed, podcast, or the GH repo to follow along for more content!

Join the YNAB Family

Signing up using my referral link will give you a 34 day trial and if you decide to purchase a subscription, YNAB will give us both a free month!

Let's look at the typical structure with minimal application code and tests.

.
├── app/
│   ├── Http/
│   │   └── Controllers/
│   │       └── MyUserController.php
│   └── Models/
│       └── User.php
└── tests/
    ├── TestCase.php
    ├── Features/
    │   └── Http/
    │       └── Controllers/
    │           └── MyUserControllerTest.php
    └── Unit/
        └── Models/
            └── UserTest.php

This issue gets exacerbated in larger applications with additional sub-directories. For example,

.
├── app/
│   ├── Http/
│   │   └── Controllers/
│   │       └── Admin/
│   │           └── Users/
│   │               └── MyUserController.php
│   └── Models/
│       └── User.php
└── tests/
    ├── TestCase.php
    ├── Features/
    │   └── Http/
    │       └── Controllers/
    │           └── Admin/
    │               └── Users/
    │                   └── MyUserControllerTest.php
    └── Unit/
        └── Models/
            └── UserTest.php

This structure further breaks down if you've chosen to break up your application within modules or domains but don't include the tests within the directories. The application and test structures could easily be five or more directories deep.

Discoverability of application files with their test partners is challenging, and it is unclear if a given application file has a partnering test.

The solution

Colocating tests is an ideal structure. The test files are elevated to the same "status" and layer as application files. This makes it very easy to notice, at a glance, whether a file has a partnering test. Further, this simplifies pull requests for reviewers since the two files are next to each other.

The structure of our application now looks like this.

.
├── app/
│   ├── Http/
│   │   └── Controllers/
│   │       ├── MyUserController.php
│   │       └── MyUserControllerTest.php
│   └── Models/
│       ├── User.php
│       └── UserTest.php
└── tests/
    └── TestCase.php

or

.
├── app/
│   ├── Http/
│   │   └── Controllers/
│   │       └── Admin/
│   │           └── Users/
│   │               ├── MyUserController.php
│   │               └── MyUserControllerTest.php
│   └── Models/
│       ├── User.php
│       └── UserTest.php
└── tests/
    └── TestCase.php

Now imagine if this app contains hundreds of different directories and files. Can you easily see if MyUserController has a test? Yes! How about the User model? Yes again!

The mechanics

Unfortunately, we cannot just move our tests into this new structure. We'll have to make some small adjustments, but they are small and straightforward.

composer.json

Add autoload.exclude-from-classmap

{
    "autoload": {
        "psr-4": {
            "App\\": "app/",
            "Database\\Factories\\": "database/factories/",
            "Database\\Seeders\\": "database/seeders/"
-        }
+        },
+        "exclude-from-classmap": [
+            "app/**/*Test"
+        ]
    }
}

phpunit.xml

Add or adjust source and testsuites.

	<source>  
	    <include>  
	        <directory suffix=".php">./app</directory>
	    </include>  
+	    <exclude>  
+	        <directory suffix="Test.php">./app</directory>
+	    </exclude>  
	</source>
    <testsuites>
-        <testsuite name="Unit">
-            <directory>tests/Unit</directory>
-        </testsuite>
-        <testsuite name="Feature">
-            <directory>tests/Feature</directory>
-        </testsuite>
+        <testsuite name="Application Test Suite">
+            <directory>./app</directory>
+        </testsuite>
    </testsuites>

tests/Pest.php

Ensure all tests have the TestCase and other uses classes available.

uses(  
    Tests\TestCase::class,  
    // Illuminate\Foundation\Testing\RefreshDatabase::class,  
-)->in('Feature');
+)->in('../');

app/Console/Kernel.php (< v11)

Add or adjust the load() method.

protected function load($paths)
{
	$paths = Arr::wrap($paths);
	$namespace = $this->app->getNamespace();

	foreach ((new Finder)->in($paths)->files() as $command) {
		$command = $namespace.str_replace(
			['/', '.php'],
			['\\', ''],
			Str::after($command->getPathname(), realpath(app_path()).DIRECTORY_SEPARATOR)
		);

+		$isTestClass = Str::endsWith($command, 'Test');

		if (
+			!$isTestClass &&
			is_subclass_of($command, Command::class) &&
			!(new ReflectionClass($command))->isAbstract()
		) {
			Artisan::starting(function ($artisan) use ($command) {
				$artisan->resolve($command);
			});
		}
	}
}

External Tooling

If your application uses a tool like Code Climate, you might need to make additional config changes. For example, within the .codeclimate.yml, add **/*Test.php within the exclude_patterns.

exclude_patterns:
+  - '**/*Test.php'

Quality of life

If you like the idea of colocating tests, but don't want the additional mess of looking at test files all the time you can enable File Nesting in PHPStorm, and other JetBrains IDEs. Once you add the .php -> Test.php mapping your files will be collapsed.

Thanks to Enzo Innocenzi for bringing that to my attention.

If you use VSCode, please check out Liam Hammett's extension.

Wrapping up

Even though colocating your tests within the app/ directory in your Laravel application is currently atypical, I highly recommend you try it out. By colocating tests alongside application code, the direct mapping of the two will become stronger, minimizing the cognitive load in your app. You will also have a simplified structure and ease of refactoring if you happen to move your modules or domains within your app or to a new one.

Please comment below or reach out via X/Twitter about what you think about colocating tests and if you've implemented it within your application.

Listen to the Podcast

The Laravel docs show this as an example

Queue::assertPushed(function (ShipOrder $job) use ($order) {
    return $job->order->id === $order->id;
});

Straightforward, right? We're trying to ensure that the ShipOrder job was pushed to the queue and the order IDs matched.

What happens when the IDs do not match, or something else happens? We get the following error

Failed asserting that false is true.

Which is not that helpful. We know that the job wasn't pushed, but was it an error with the IDs matching, or something else? This issue gets exacerbated when you have multiple conditions that need to pass.

Queue::assertPushed(function (ShipOrder $job) use ($order) {
    return $job->order->id === $order->id
        && $job->order->second === $order->second
        && $job->order->third === $order->third
        && $job->order->fourth === $order->fourth;
});

Now, if any of those conditions fail, then we'll still get the same PHPUnit error message.

Failed asserting that false is true.

There is no way to know which condition failed easily. You can remove one condition at a time, but that's a time-consuming process. There's a better way.

Instead of using chained conditions, we can use assertions for each of the comparisons. Then we can manually return true if all of the conditions are true.

Queue::assertPushed(function (ShipOrder $job) use ($order) {
    $this->assertSame($job->order->id, $order->id);
    $this->assertSame($job->order->second, $order->second);
    $this->assertSame($job->order->third, $order->third);
    $this->assertSame($job->order->fourth, $order->fourth);

    return true;
});

Now, if any of these assertions fail, PHPUnit will be able to narrow in on the exact line number that failed - making it quicker to debug and getting you on your way.

Failed asserting that X is identical to Y

# ---

Failed asserting that 456 is identical to 123.
 /tests/PathToFileTest.php:62

In conclusion, by using assertions for each comparison, you can quickly pinpoint and resolve issues in your code. Leveraging Laravel mocks effectively can significantly improve your productivity as a developer.

Don't forget to share this article with your fellow developers to help them optimize their Laravel testing experience!

Tips, Tricks, and Good Practices

with

Laravel's Eloquent

Presented by Chris Gmyr

What is Laravel?

Laravel is a modern PHP framework that helps you create applications using simple, expressive syntax as well as offers powerful features like an ORM, routing, queues, events, notifications, simple authentication...

...and so much more!

What is Eloquent?

The Eloquent ORM included with Laravel provides a beautiful, simple ActiveRecord implementation for working with your database. Each database table has a corresponding "Model" which is used to interact with that table. Models allow you to query for data in your tables, as well as insert new records into the table.

https://laravel.com/docs/5.6/eloquent

The Basics

A Model

class Post extends Model
{
    // look Ma, no code!
}

- id
- title
- created_at
- updated_at

$post = Post::find(1);

Artisan Goodies

php artisan make:model Product

Artisan Goodies

php artisan make:model Product -mcr

-m will create a migration file -c will create a controller -r will indicate that controller should be resourceful

Cruddy

Creating

$user = new User();
$user->first_name = 'Chris';
$user->email = 'cmgmyr@gmail.com';
$user->save();

Creating

$user = User::create([
    'first_name' => 'Chris',
    'email' => 'cmgmyr@gmail.com',
]);

Note: $fillable/$guarded properties

Updating

$user = User::find(1);
$user->email = 'me@chrisgmyr.com';
$user->save();

Updating

$user = User::find(1);
$user->update([
    'email' => 'me@chrisgmyr.com',
]);

Note: $fillable/$guarded properties

Updating

$user = User::find(1);
$user->fill([
    'email' => 'me@chrisgmyr.com',
]);
$user->save();

Note: $fillable/$guarded properties

Deleting

$user = User::find(1);
$user->delete();

Deleting

User::destroy(1);
User::destroy([1, 2, 3]);
User::destroy(1, 2, 3);

"or" helper methods

User::findOrFail(1);

$user->saveOrFail(); // same as save(), but uses transaction

User::firstOrCreate([ /* attributes */]);

User::updateOrInsert([/* attributes to search */], [/* attributes to update */]);

Querying

Querying

$users = User::get(); // User::all()
$user  = User::where('id', 1)->first();
$user  = User::find(1);
$user  = User::findOrFail(1);
$users = User::find([1, 2, 3]);
$users = User::whereIn('id', [1, 2, 3])->get();

$users = User::where('is_admin', true)
    ->where('id', '!=', Auth::id())
    ->take(10)
    ->orderBy('last_name', 'ASC')
    ->get();

Chunking

User::chunk(50, function ($users) {
    foreach ($users as $user) {
        //
    }
});

Collections

For Eloquent methods like all() and get() which retrieve multiple results, an instance of Illuminate\Database\Eloquent\Collection will be returned.

$admins = $users->filter(function ($user) {
    return $user->is_admin;
});

Raw query methods

Product::whereRaw('price > IF(state = "NC", ?, 100)', [200])
    ->get();

Post::groupBy('category_id')
    ->havingRaw('COUNT(*) > 1')
    ->get();

Customer::where('created_at', '>', '2016-01-01')
    ->orderByRaw('(updated_at - created_at) desc')
    ->get();

Relationships

Relationships

hasOne() // User has one Address
belongsTo() // Address belongs to User
hasMany() // Post has many Comment
belongsToMany() // Role belongs to many User
hasManyThrough() // Country has many Post through User

// Use single table
morphTo() // Comment can be on Post, Video, Album
morphMany() // Post has many Comment

// Use pivot table
morphToMany() // Post has many Tag
morphedByMany() // Tag has many Post

Relationships

class Video extends Model
{
    public function comments()
    {
        return $this->hasMany(Comment::class);
    }
}

$video = Video::find(1);
foreach ($video->comments as $comment) {
    // $comment->body
}

Relationships

class Video extends Model
{
    public function comments()
    {
        return $this->hasMany(Comment::class);
    }
}

$video = Video::find(1);
foreach ($video->comments()->where('approved', true)->get() as $comment) {
    // $comment->body
}

Default conditions and ordering

class Video extends Model
{
    public function comments()
    {
        return $this->hasMany(Comment::class)
        ->where('approved', true)
        ->latest();
    }
}

Default conditions and ordering

class Video extends Model
{
    public function comments()
    {
        return $this->hasMany(Comment::class);
    }

    public function publicComments()
    {
        return $this->comments()
        ->where('approved', true)
        ->latest();
    }
}

Default Models

Default models can be used with belongsTo(), hasOne(), and morphOne() relationships.

Default Models

{{ $post->author->name }} // error if author not found

class Post extends Model
{
    public function author()
    {
        return $this->belongsTo(User::class);
    }
}

Default Models

{{ $post->author->name ?? '' }} // meh

class Post extends Model
{
    public function author()
    {
        return $this->belongsTo(User::class);
    }
}

Default Models

{{ $post->author->name }} // better!

class Post extends Model
{
    public function author()
    {
        return $this->belongsTo(User::class)->withDefault();
    }
}

Default Models

class Post extends Model
{
    public function author()
    {
        return $this->belongsTo(User::class)->withDefault([
            'name' => 'Guest Author',
        ]);
    }
}

Events

The retrieved event will fire when an existing model is retrieved from the database. When a new model is saved for the first time, the creating and created events will fire. If a model already existed in the database and the save() method is called, the updating / updated events will fire. However, in both cases, the saving / saved events will fire.

https://laravel.com/docs/5.6/eloquent#events

Events

class User extends Model
{
    protected $dispatchesEvents = [
        'saved' => UserSaved::class,
        'deleted' => UserDeleted::class,
    ];
}

Observers

php artisan make:observer UserObserver --model=User

class ModelObserverServiceProvider extends ServiceProvider
{
    public function boot()
    {
        User::observe(UserObserver::class);
    }
}

Observers

class UserObserver
{
    public function created(User $user)
    {
    }

    public function updated(User $user)
    {
    }

    public function deleted(User $user)
    {
    }
}

boot() method

class Post extends Model
{
    public static function boot()
    {
        parent::boot();

        self::creating(function ($model) {
            $model->uuid = (string) Uuid::generate();
        });
    }
}

Bootable Trait

class Post extends Model
{
    use HasUuid;
}

trait HasUuid
{
    public static function bootHasUuid()
    {
        self::creating(function ($model) {
            $model->uuid = (string) Uuid::generate();
        });
    }

    // more uuid related methods
}

Helper Methods

Increments and Decrements

$post = Post::find(1);
$post->stars++;
$post->save();

$post->stars--;
$post->save();

Increments and Decrements

$post = Post::find(1);
$post->increment('stars'); // add 1
$post->increment('stars', 15); // add 15

$post->decrement('stars'); // subtract 1
$post->decrement('stars', 15); // subtract 15

Aggregates

$count = Product::where('active', 1)->count();
$min   = Product::where('active', 1)->min('price');
$max   = Product::where('active', 1)->max('price');
$avg   = Product::where('active', 1)->avg('price');
$sum   = Product::where('active', 1)->sum('price');

Check if Records Exist

Instead of count(), you could use...

User::where('username', 'cmgmyr')->exists();

User::where('username', 'cmgmyr')->doesntExist();

Model State

$model->isDirty($attributes = null);
$model->isClean($attributes = null);
$model->wasChanged($attributes = null);
$model->hasChanges($changes, $attributes = null);
$model->getDirty();
$model->getChanges();

//Indicates if the model exists.
$model->exists;

//Indicates if the model was inserted during the current request lifecycle.
$model->wasRecentlyCreated;

"Magic" where()

$users = User::where('approved', 1)->get();
$users = User::whereApproved(1)->get();

$user = User::where('username', 'cmgmyr')->get();
$user = User::whereUsername('cmgmyr')->get();

$admins = User::where('is_admin', true)->get();
$admins = User::whereIsAdmin(true)->get();

Super "Magic" where()

User::whereTypeAndStatus('admin', 'active')->get();
User::whereTypeOrStatus('admin', 'active')->get();

https://twitter.com/themsaid/status/1029731544942952448

Dates

User::whereDate('created_at', date('Y-m-d'));
User::whereDay('created_at', date('d'));
User::whereMonth('created_at', date('m'));
User::whereYear('created_at', date('Y'));

when() to eliminate conditionals

$query = Author::query();

if (request('filter_by') == 'likes') {
    $query->where('likes', '>', request('likes_amount', 0));
}

if (request('filter_by') == 'date') {
    $query->orderBy('created_at', request('ordering_rule', 'desc'));
}

when() to eliminate conditionals

$query = Author::query();

$query->when(request('filter_by') == 'likes', function ($q) {
    return $q->where('likes', '>', request('likes_amount', 0));
});

$query->when(request('filter_by') == 'date', function ($q) {
    return $q->orderBy('created_at', request('ordering_rule', 'desc'));
});

replicate() a Model

$invoice = Invoice::find(1);
$newInvoice = $invoice->replicate();
$newInvoice->save();

Pagination

// 1, 2, 3, 4, 5...
$users = User::where('active', true)->paginate(15);

// Previous/Next
$users = User::where('active', true)->simplePaginate(15);

// In Blade
{{ $users->links() }}

Pagination to Json

{
   "total": 50,
   "per_page": 15,
   "current_page": 1,
   "last_page": 4,
   "first_page_url": "https://my.app?page=1",
   "last_page_url": "https://my.app?page=4",
   "next_page_url": "https://my.app?page=2",
   "prev_page_url": null,
   "path": "https://my.app",
   "from": 1,
   "to": 15,
   "data":[
        {
            // Result Object
        },
        {
            // Result Object
        }
   ]
}

Model Properties

protected $table = 'users';
protected $fillable = ['first_name', 'email', 'password']; // create()/update()
protected $dates = ['created', 'deleted_at']; // Carbon
protected $appends = ['full_name', 'company']; // additional JSON values
protected $casts = ['is_admin' => 'boolean', 'options' => 'array'];

protected $primaryKey = 'uuid';
public $incrementing = false;
protected $perPage = 25;
const CREATED_AT = 'created';
const UPDATED_AT = 'updated';
public $timestamps = false;

...and more!

Overriding updated_at

$product = Product::find(1);
$product->updated_at = '2020-01-01 10:00:00';
$product->save(['timestamps' => false]);

Primary Key Methods

$video = Video::find(1);
$video->getKeyName(); // 'id'
$video->getKeyType(); // 'int'
$video->getKey(); // 1

Accessors/Mutators

class User extends Model
{
    public function setFirstNameAttribute($value)
    {
        $this->attributes['first_name'] = strtolower($value);
    }
    public function setLastNameAttribute($value)
    {
        $this->attributes['last_name'] = strtolower($value);
    }
}

Accessors/Mutators

class User extends Model
{
    public function getFirstNameAttribute($value)
    {
        return ucfirst($value);
    }
    public function getLastNameAttribute($value)
    {
        return ucfirst($value);
    }
    public function getEmailAttribute($value)
    {
        return new Email($value);
    }
    public function getFullNameAttribute()
    {
        return "{$this->first_name} {$this->last_name}";
    }
}

Accessors/Mutators

$user = User::create([
    'first_name' => 'Chris', // chris
    'last_name' => 'Gmyr', // gmyr
    'email' => 'cmgmyr@gmail.com',
]);

$user->first_name; // Chris
$user->last_name; // Gmyr
$user->email; // instance of Email
$user->full_name; // 'Chris Gmyr'

To Array/Json

$user = User::find(1);
return $user->toArray();
return $user->toJson();

You can also return $user from a controller method and it will automatically return JSON.

Appending Values to JSON

class User extends Model
{
    protected $appends = ['full_name']; // adds to toArray()

    public function getFullNameAttribute()
    {
        return "{$this->first_name} {$this->last_name}";
    }
}

// or...
return $user->append('full_name')->toArray();
return $user->setAppends(['full_name'])->toArray();

Local Scopes

$posts = Post::whereNotNull('published_at')
    ->where('published_at', '<=', Carbon::now())
    ->latest('published_at')
    ->get();

Local Scopes

class Post extends Model
{
    public function scopePublished($query)
    {
        return $query->whereNotNull('published_at')
            ->where('published_at', '<=', Carbon::now())
            ->latest('published_at');
    }
}

Local Scopes

$posts = Post::published()->get();

Single Table Inheritance

Single Table Inheritance

$admins = User::where('is_admin', true)->get();
$customers = User::where('is_admin', false)->get();

Single Table Inheritance

class User extends Model
{
    public function scopeAdmin($query)
    {
        return $query->where('is_admin', true);
    }
    public function scopeCustomer($query)
    {
        return $query->where('is_admin', false);
    }
}

$admins = User::admin()->get();
$customers = User::customer()->get();

Single Table Inheritance

class Admin extends User
{
    protected static function boot()
    {
        parent::boot();
        static::addGlobalScope(function ($query) {
            $query->where('is_admin', true);
        });
    }
}

Single Table Inheritance

class Customer extends User
{
    protected static function boot()
    {
        parent::boot();
        static::addGlobalScope(function ($query) {
            $query->where('is_admin', false);
        });
    }
}

Single Table Inheritance

$admins = Admin::get();
$customers = Customer::get();

Single Table Inheritance

Read more:

• https://twitter.com/cmgmyr/status/885204646498893824

• https://tighten.co/blog/extending-models-in-eloquent

Default Model Data

Default Model Data

Schema::create('users', function (Blueprint $table) {
    $table->increments('id');
    $table->string('name');
    $table->string('email')->unique();
    $table->string('password');
    $table->string('role')->default('user'); // moderator, admin, etc
    $table->rememberToken();
    $table->timestamps();
});

Default Model Data

class User extends Model
{
    protected $fillable = [
        'name', 'email', 'password', 'role'
    ];
}

Default Model Data

$user = new User();
$user->name = 'Chris';
$user->email = 'cmgmyr@gmail.com';
$user->password = Hash::make('p@ssw0rd');

// $user->role is currently NULL

$user->save();

$user->role; // 'user'

Default Model Data

Remove ->default('user');

Schema::create('users', function (Blueprint $table) {
    $table->increments('id');
    $table->string('name');
    $table->string('email')->unique();
    $table->string('password');
    $table->string('role'); // moderator, admin, etc
    $table->rememberToken();
    $table->timestamps();
});

Default Model Data

Set $attributes

class User extends Model
{
    protected $fillable = [
        'name', 'email', 'password', 'role'
    ];

    protected $attributes = [
        'role' => 'user',
    ];
}

Default Model Data

$user = new User();
$user->name = 'Chris';
$user->email = 'cmgmyr@gmail.com';
$user->password = Hash::make('p@ssw0rd');

// $user->role is currently 'user'!

$user->save();

$user->role; // 'user'

Default Model Data

$user = new User();
$user->name = 'Chris';
$user->email = 'cmgmyr@gmail.com';
$user->password = Hash::make('p@ssw0rd');
$user->role = 'admin'; // can override default
$user->save();

$user->role; // 'admin'

Default Models

Remember our previous example?

class Post extends Model
{
    public function author()
    {
        return $this->belongsTo(User::class)->withDefault([
            'name' => 'Guest Author',
        ]);
    }
}

Default Models

We no longer need to provide a name, use the User $attributes property!

class Post extends Model
{
    public function author()
    {
        return $this->belongsTo(User::class)->withDefault();
    }
}

Default Model Data

class User extends Model
{
    protected $fillable = [
        'name', 'email', 'password', 'role'
    ];

    protected $attributes = [
        'name' => 'Guest Author',
        'role' => 'user',
    ];
}

Default Model Data

Watch Colin DeCarlo's - Keeping Eloquent Eloquent from Laracon US 2016

https://streamacon.com/video/laracon-us-2016/colin-decarlo-keeping-eloquent-eloquent

Sub-Queries

$customers = Customer::with('company')
    ->orderByName()
    ->paginate();

Get latest interactions?

<p>{{ $customer
    ->interactions()
    ->latest()
    ->first()
    ->created_at
    ->diffForHumans() }}</p>

Sub-Queries

public function scopeWithLastInteractionDate($query)
{
    $subQuery = \DB::table('interactions')
        ->select('created_at')
        ->whereRaw('customer_id = customers.id')
        ->latest()
        ->limit(1);

    return $query->select('customers.*')->selectSub($subQuery, 'last_interaction_date');
}

$customers = Customer::with('company')
    ->withLastInteractionDate()
    ->orderByName()
    ->paginate();

<p>{{ $customer->last_interaction_date->diffForHumans() }}</p>

Sub-Queries

Jonathan Reinink's Laracon 2018 Online Talk - Advanced Querying with Eloquent

https://github.com/reinink/laracon2018

Resources

• https://laravel.com/docs/5.6/eloquent

• https://laravel-news.com/eloquent-tips-tricks

• https://twitter.com/themsaid/status/1029731544942952448

• https://twitter.com/cmgmyr/status/885204646498893824

• https://tighten.co/blog/extending-models-in-eloquent

• https://streamacon.com/video/laracon-us-2016/colin-decarlo-keeping-eloquent-eloquent

• https://github.com/reinink/laracon2018

• https://eloquentbyexample.com

Thank you!

Please say "hi"

twitter.com/cmgmyr

github.com/cmgmyr

chrisgmyr.com

My project has two main branches: develop and master. In Envoyer I have two projects, one for dev.project.com which uses the develop branch and the other for project.com which uses the master branch.

Here is the final result of the circle.yml file. Let's work through each of the sections below.

Section 1: Defaults

By leveraging YAML anchors we can set a group of defaults that will be used for all of our later jobs. For now, this includes our

• working directory

• chosen CircleCI docker image

Section 2: Jobs

In this file we have three jobs: build (and test), deploy_develop, and deploy_master.

Our build job

1. Imports the defaults

2. Sets environment variables

3. Checks out the repo's code

4. Restores composer cache, if available

5. Runs composer install

6. Saves a new composer cache

7. Runs the test suite with PHPUnit

Our "deploy" jobs:

1. Imports the defaults

2. Pings Envoyer to deploy the project

Section 3: Workflows

Now that we have our jobs set up, we need to implement workflows to pull everything together. Workflows are optional, but they can come in handy depending on what you'd like to do with your project.

In this example, we only need one workflow notify_deploy which will notify Envoyer that we want to deploy a specific branch.

Within the workflow, you'll notice that we are listing all of our jobs: build, deploy_develop, and deploy_master.

We start off running our build job, and if that is successful, we'll move forward with our deploy jobs. Each deploy job requires the build to run first; then we'll check if the version branch matches the filter on the workflow. So deploy_develop is only run on the develop branch and deploy_master is only run on the master branch.

By limiting the filters to only the develop and master branches we can guarantee that we're only deploying those specific branches, but the build job will run on all branches (bug, hotfix, and feature branches), which is needed for pull requests.

Wrapping Up

Once we merge a branch into either develop or master CircleCI will build and notify Envoyer to deploy if successful. In our CircleCI dashboard, you'll now see a workflow similar to this.

You'll also need to make sure you turn off the "Deploy When Code Is Pushed" option in your Enoyer project.

Learn More

This is only scratching the surface of what you can do with CircleCI builds and workflows. I encourage you to look through the documentation and example projects to see what you can implement in your projects.

• 2.0 Documentation

• Sample Projects

• Workflow Documentation

• YAML 1.2 Spec

Please note - CircleCI nor Envoyer/Laravel paid me to write this article, I'm just a happy customer.

“I read through the entire codebase of Basecamp 3 and try to make things better that I don’t think are good enough, or revisit decisions that we’ve made earlier that I think now I have a better idea of how to do”

There’s a lot in that statement, so let’s unpack it.

First, he reads through the entire codebase! The fact that he does this shows great care for what he does and believes in.

Second, he makes changes where he doesn’t feel like the code is good enough. While many of us have ideas about what is “good” or not, I’m sure we’ve all gone back through older code and just known when we’ve seen it.

Lastly, he revisits past decisions that can be better handled now. Programming is an ever-changing space and developers are enhancing their skills and knowledge on a daily basis. So why shouldn’t our code reflect our most up-to-date understandings?

DHH isn’t the only one who does this though. Taylor Otwell (creator of Laravel) does something similar.

%[https://twitter.com/taylorotwell/status/818863355066798080]

Going through code and documentation that you’ve already worked on isn’t the most glamorous job, even quite tedious, but necessary. It’s silly to think that something that was done years, weeks, or even days ago is still “good enough” for today.

I know I’ve fallen into the habit of not revisiting what I’ve worked on long ago — and maybe some projects don’t need it. However, if the public is using it — like in a current site, app, or package it might be time to take a look through for improvements.

Next Steps

I’m going to take a hard look at my messenger package (which needs some love) as well as a few side projects that I haven’t worked on in a while. What projects are you going to look at? Let’s share some successes. Send me a tweet or screenshot on Twitter, or leave a comment below.

$users = User::all();

But what happens when you want to sort your users?

As newcomers to the framework, I feel like most are too excited to “jump in and build something” instead of learning more about it. (But who can blame them, right?!?) So something like this would happen:

$users = User::all()->orderBy('name', 'ASC');

# BadMethodCallException with message 'Method orderBy does not exist.'

// or

$users = User::orderBy('name', 'ASC')->all();

# BadMethodCallException with message 'Call to undefined method Illuminate\Database\Query\Builder::all()'

Forget about all()

In my experience, I’ve never needed an unordered dump of data in an application.

Note that all() is a convenience method for get() but does not allow you to chain additional methods. Take a look:

public static function all($columns = ['*'])
{
    return (new static)->newQuery()->get
        (is_array($columns) ? $columns : func_get_args()
    );
}

By using get() you’ll be able to achieve the desired results.

$users = User::orderBy('name', 'ASC')->get();

// and

$users = User::where('email', 'LIKE', '%@gmail.com')  
    ->orderBy('name', 'ASC')->get();

So any time you reach for the all() method, I highly recommend using get() instead.

On a current project, I had to figure out how to grab a ton of data from an API (Facebook) that had dependent data — meaning the script cannot proceed to get new data before it’s done with the current data set. This typically ended up being an ID that was needed for the next call as well as some data that needed to be processed.

In pseudo-code, this would look something similar to:

$a = $get->a();  
$b = $get->b($a);   
$c = $get->c($b);   
$this->doSomethingWith($c);

As you can see, the code should not move ahead without processing the previous data. In my situation, I had to process all of the “A” jobs, then all of the “B” jobs, and so on, before continuing.

The data that I needed was quite large and each round needed a good amount of processing before continuing to the next step. Luckily this is where Laravel and its queue system stepped in to help!

Laravel lets you specify a dynamic queue name along with a job, like so:

dispatch((new JobA($data))->onQueue('a'));

In my application in each “JobA” a “JobB” would be called. For each “JobB” a “JobC” would be called. At the end of “JobC”, we’d do some additional work on the whole data collection. So you’d get something like this:

// In Controller  
dispatch((new JobA($data))->onQueue('a'));
 
// In JobA  
dispatch((new JobB($data))->onQueue('b'));
 
// In JobB  
dispatch((new JobC($data))->onQueue('c'));
 
// In JobC  
dispatch((new JobFinish($data))->onQueue('finish'));

I wanted to make sure we ran all of the jobs in order, and only continued onto the next set of jobs once the current batch was finished. This was very important since the application could easily have 20, or so, JobAs, 50 JobBs, and hundreds of JobCs.

In my supervisor config file, I added something similar to this:

[program:artisan-queue]  
command = php artisan queue:work --queue=a,b,c,finish

Now all of the jobs on the “a” queue would have to finish before the “b” jobs would start, and “c” would wait for “b”, and so on.

So there you have it — a prioritized queuing system in only a few lines of code!

The Laravel Queue system is very robust, and if you haven’t used it, I’d highly recommend trying it in your next project. You can read more about the queue system and queue priorities here.

Image manipulation is hard. Handling images in the long term is even harder. Here at Dose we have a lot of images and a number of ways to serve them. We have Android and iOS apps, as well as completely responsive websites, so each image has the potential of getting manipulated multiple times depending on the device, orientation, and how we optimize an image for a certain platform.

Handling this ourselves

We used to have an internal image service that took an author’s uploaded image and handled some pre-processing. On upload, we’d make sure it’s within a max width and encoded correctly. Animated GIF? We’d have to convert the file to MP4 and WEBM also. This would end up adding extra load on our servers and more space taken in our S3 account.

When an asset was requested with a certain height/width combination we’d check to see if we had it in our S3 bucket. If not we’d return the master asset and kick off a queue job to create it then add it to the bucket for the next request. As you can imagine, the usage was huge. Want to change the article promo image from 500px width to 475px width? That would invalidate all previous images and we’d need to recreate all new images the next time they were requested. What a mess!

If your product is not image manipulation, then don’t do this yourself. Services like Cloudinary do this much more efficiently and much better than you will, so use them. And if you’re worried about the cost, think about how much it’ll cost you in development and upkeep, as well as hosting, storage, and delivery costs. More on that later.

Getting Started with Cloudinary

Cloudinary is the market leader in providing a comprehensive cloud-based image management solution. Cloudinary is being used by tens of thousands of web and mobile application developers all around the world, from small startups to large enterprises. We are here to cover your every image-related need.

Source

Cloudinary has taken the massive task of handling and manipulating images and broke it down to something very simple — URL manipulation.

Take the following image url:

http://res.cloudinary.com/demo/image/upload/sample.jpg

Let’s say we’d like to resize this to a max width of 300px, you’d get

http://res.cloudinary.com/demo/image/upload/w_300/sample.jpg

How about we restrict the height to 150px

http://res.cloudinary.com/demo/image/upload/w_300,h_150/sample.jpg

Well, we got the size that we wanted, but it looks pretty bad right now. We’ll need to make some adjustments. Let’s crop it to fit the space that we need

http://res.cloudinary.com/demo/image/upload/w_300,h_150,c_fit/sample.jpg

Yeah, this looks a lot better!

Combining Transformations

Combining transformations is just as straightforward as single transformations, all you have to do is add another segment to the URL

http://res.cloudinary.com/demo/image/upload/w_300,h_150/c_crop,w_50,h_50,x_100,y_75/sample.jpg

So with this example we are:

1. Resizing the image to 300px X 150px then

2. Cropping the image to be 50px X 50px and moving the X, Y point to 100, 75 in order to focus in on the yellow part of one of the flowers.

Learn more

File Type Transformations

This is by far one of the most powerful features of Cloudinary. Say we are uploading a GIF, but want to optimize it and change it to a MP4.

http://res.cloudinary.com/demo/image/upload/kitten_fighting.gif

would turn into

http://res.cloudinary.com/demo/image/upload/kitten_fighting.mp4

Just by changing the extension of the file in the URL will make this conversion for you. No more keeping track of different file references or different hashes. Only ask for a different file extension!

Learn more

Programmatic SDKs

It’s all well and good that we can convert any file asset via a URL, but if you want to make these changes more programmatically, you can use one of their many SDKs. For this example, we’ll be using their PHP SDK

In my initial example, we ended with the final URL of

http://res.cloudinary.com/demo/image/upload/w_300,h_150,c_fit/sample.jpg

for our image. In the PHP SDK, we’d be able to do

$transformations = ['width' => 100, 'height' => 150, 'crop' => 'fill'];  
$url = cloudinary_url('sample.jpg', $transformations);

and multiple transformations, like our second example, would look like

$transformations = ['transformation' => [  
   ['width' => 300, 'height' => 150],  
   ['width' => 50, 'height' => 50, 'crop' => 'crop', 'x' => 100, 'y' => 75]  
]];  
$url = cloudinary_url('sample.jpg', $transformations);

Migrating to Cloudinary

Cloudinary has taken the hard work out of migrating assets over to their platform. There are currently a few options to choose from.

1. Add a full URL to your current image and Cloudinary will automatically pull this in for you.

2. Set up an auto upload mapping to your S3 bucket. When this “virtual” directory is requested, it will pull the asset from your bucket into your Cloudinary account.

Learn more

Why We Chose Cloudinary

Before moving forward with anything new, we went through a diligent research period where we were looking at a handful of SaaS options as well as possibly redesigning our current in-house system. As we worked through the options one thought became very clear — handling our own system didn’t make sense. There are a good handful of services that are more cost-effective and feature-rich than something that we could ever make. Our business is not handling images. Just the cost of AWS instances, storage, and CDN fees were tens of thousands of dollars per month. Now take developer costs to maintain and add to the system, and it adds up quickly. But even more importantly, each time something would break or need to be added to our image service, it would take attention away from more important tasks or services.

After evaluating a number of image and CDN services, we moved forward with Cloudinary for a number of reasons:

1. They were very attentive. They set up multiple meetings with their sales and tech teams to answer all of our questions and to help get us the best price for the resources we need. Even now, after the “sale”, they continue to reach out personally to share new features that haven’t been published yet and check in on analytics and our performance.

2. Price. Their price is a LOT cheaper than running all of our instances, storage, and CDN through AWS.

3. Features. The flexibility and amount of features they have for image handling is impressive and there would be no realistic way that we’d have the bandwidth in order to make similar features ourselves. Their system makes it very easy to make adjustments on the fly to see what combinations perform, and look better.

4. Uptime. Since the majority of our content is showing images, if the images aren’t available we aren’t able to provide the value to our users that we need to. Since moving to Cloudinary, our image uptime has been spectacular and image responsiveness has been significantly faster too.

Summary

We have simplified our image handling process over the last 6 months since starting to work with Cloudinary. They can easily handle our 4.5 Million images (and counting) and over 1.4 Billion requests per month. They also provide insight to help us further improve our performance on our sites. Site performance and user experience are not taken lightly, so we are very happy with our decision to utilize their services. Some other helpful points are

• Great documentation

• A bunch of add-on plugins like JPEGmini and Imagga

• Tons of transformation options like watermarks, custom pixelation, and even PDF to image conversion

• Automatic responsive image handling

• Video management and transformations

• Automatic backups to S3

• Newly introduced “Auto Everything”

As an aside, we’re just sharing this article and information as happy customers. Cloudinary did not commission this article nor give us any incentive for writing it. We just want to share our experience.