pre_production branch and production inventory.
codex/WREA-email-nurture-phases-0-1 in /Users/tr/dev/wrea-multi-project/wrea-email-nurture. The build now includes address suppression and one-click unsubscribe, live eligibility on legacy marketing paths, campaign/step/enrolment/attempt state, consent snapshots, deterministic holdout, audience preview, reliable sending with explicit ambiguity, property-local send windows, hourly throttles, dry-run enrol/retry/reconciliation commands, a ten-minute scheduler, and staff/admin CRM controls. Migration rollback/reapply passed and the integrated nurture regression passed 76 tests / 326 assertions. No production changes and no external email sends were made.
What remains before any live customer pilot: approve the production workflow keep/defuse register; run and reconcile the production suppression backfill; select and approve the two email templates and visible sender; record the recent-enquiry consent approval in CRM; complete a real allowlisted Gmail/Outlook/iCloud header/body-unsubscribe test; then run the separately approved 15-lead micro-cohort. These are release and business inputs, not unfinished sending architecture.
1. Context and strategy (why this shape)
WREA leads convert or die fast: in the last 180 days, 2,656 leads reached meeting-agents and 2,366 of those (89%) did so within 24 hours; 2,567 within 72 hours; 2,627 within 7 days. Failed outcomes are equally fast (3,022 within 24h). A long fixed drip therefore cannot be the conversion engine. The strategy this spec implements is:
- Two send systems, not six lanes. Rescue (short conversion support, days 0–7, for leads that stall) and Utility (one monthly per-property/suburb send built from modules — market data, property-value refresh, decision-relief content — where a genuine market event replaces that month's regular send rather than adding to it). Phase 0 showed the "local utility" and "market event" audiences are ~identical (10,083 vs 10,104 contacts); running them as separate systems guarantees frequency conflicts.
- A suppression/eligibility layer under everything. One function decides marketability; preview and send share that code path.
- Email capture is the binding constraint, not email cleverness. 16,455 of 30,505 property-owner contacts (730d) have no valid email; 66% of rescue candidates are unreachable. The new property value tool is the capture engine and must be instrumented from day one (§12).
- ROI = downstream CRM movement vs holdout, never opens. Campaign identity travels in SendGrid custom args and lands as columns on
msg_emails, joined to milestone timestamps.
Full lane sizing and exclusion numbers are in the Appendix (§15) and in artifacts/wrea/email-nurture-phase0/2026-06-30/phase0-findings.md.
2. Executive decisions taken
These decisions are locked for Phase 0 and Phase 1. Future changes require an explicit spec revision; Felix should not silently substitute a different behaviour during implementation.
| Decision | Rationale |
|---|---|
D1. Nurture sending does NOT run through the workflow engine (triggers_actions_queue). | The workflow engine supports week/month delays, but it has no durable journey state, consent evidence, holdout, audience preview or suppression contract; re-triggering can invalidate an in-flight sequence and archived queue rows are poor reporting state. A dedicated email-nurture:tick runner driven by email_enrolments.next_send_at is previewable and stoppable. Workflows may enrol a journey, never send its steps. |
| D2. One live eligibility decision, enforced at preview and immediately before send. | EligibilityService reads current address suppression, contact preferences, blacklist and frequency history. No materialised eligibility cache is built. Marketing-class legacy paths also call the same gate at send time, closing the mass-email mid-drain unsubscribe hole. |
| D3. Campaign identity and communication class are first-class. | campaign_id / enrolment_id / step_no / communication_class / ip_pool are stamped on msg_emails and outbound custom args. communication_class — not IP pool — decides whether marketing suppression/frequency rules apply. Custom args are persisted by the webhook but the existing per-email helper must be wired. |
| D4. Holdout is deterministic and mandatory (default 20%) on every recurring campaign. | crc32(contact_id . campaign_key) % 100 < holdout_pct. Opens are inflated by Apple MPP, and rescue emails will otherwise claim credit for conversions staff would have won anyway. Lift vs holdout is the only success metric. |
| D5. Existing commercial workflow journeys are classified and deliberately kept or defused in Phase 0. | The 6 Week Home Seller Plan has ten staggered actions over six weeks, not ten immediate sends, but it is live and unsuppressed. Defusing means disabling new trigger execution and pending queue rows. The same inventory/decision covers Seller Guide and Home Selling Masterclass workflows; soft-deleted rows are excluded. |
| D6. Utility = one monthly send; events replace, never stack. | Global frequency cap: max 2 marketing emails per contact per rolling 14 days, across ALL campaigns, enforced in eligibility. In an event month, the event email consumes the monthly slot. |
| D7. Stage-branched copy is mandatory for utility/value sends. | Phase 0 samples mixed failed, meeting-agents and agent-appointed records in one lane. Failed/unresponsive sellers must not get "ready to meet agents?" copy — template selection branches on property stage. |
| D8. Property value tool is specced as the email-capture engine with verified email + scoped consent (§12). | Do not build around today's explicit tool tags (14 properties / 10 contacts). Instrument the new tool properly instead. |
| D9. Suppression is address-level; preferences and intent can also be contact-level. | A normalized address suppression blocks that address across duplicate contacts and unresolved webhooks. Unsubscribe/spam also applies the canonical contact tag and synchronously stops active journeys for the resolved contact; bounce/block/invalid/dropped remain address-only. |
| D10. Every nurture step has visible attempt state. | email_send_attempts is the outbox/idempotency record. A unique enrolment+step claim prevents duplicates. Deterministic pre-send failures are retryable; an exception during the provider call is ambiguous and never auto-retried. |
| D11. Consent evidence is snapshotted per enrolment. | The campaign declares the required basis/scope; each enrolment records the actual source, source id and timestamp. Missing evidence fails closed with NO_CONSENT_EVIDENCE. P1 rescue uses the recent property enquiry record; PVT consent events plug in later without a new consent table. |
| D12. Journey validity is checked before every step. | Marketability is not enough: a rescue lead must still be stalled, within the allowed time window and outside failed/closed stages. Progressed rows stop as PROGRESSED; overdue rows stop as EXPIRED. Resuming a campaign never drains stale sends. |
3. Requirements
3.1 Functional requirements
| ID | Requirement | Phase |
|---|---|---|
| R1 | A single live eligibility decision (can this normalized address receive this marketing communication right now?) with machine-readable reasons, used by preview and immediately before dispatch. | P0 |
| R2 | SendGrid suppression events, one-click POST, body unsubscribe and staff/manual actions durably update address-level suppression. Unresolved webhook events still suppress the recipient address. | P0 |
| R3 | Every automated email path has an explicit communication_class. Nurture, mass email and vendor digest are marketing by construction; active workflow email actions are inventoried/classified before Phase 0 exits. Marketing-class sends gate again at dispatch. | P0 |
| R4 | Commercial legacy workflows have an explicit keep/defuse decision. A defused workflow cannot enrol new recipients and has no pending executable queue rows. | P0 |
| R5 | Campaigns exist as named, versioned records with lane, status (draft/testing/active/paused/archived), steps, templates, throttle, holdout %, and a stop/pause control that takes effect within one tick. | P1 |
| R6 | Every enrolled contact/property has visible journey state: campaign, step, last send, next send, status, exclusion reason. | P1 |
| R7 | Audience preview before activation: candidate count, excluded count by reason, recent-staff-touch conflicts, sample rows — from the same query the enrolment uses. Includes the outstanding rescue overlap join (137 email-qualified × 239 recent-touch). | P1 |
| R8 | Rescue campaign live: 2 steps (+24h, +72h from lead creation) for still-stalled high-intent leads, with 14-day human staff-touch suppression, consent-evidence snapshot, 48h overdue expiry and holdout. | P1 |
| R9 | Internal test mode only sends to an internal allowlist. Every external marketing template has sender identity, a functional body unsubscribe and one-click headers; unsubscribe stops remaining steps synchronously. | P1 |
| R10 | Lane performance report: sent/delivered/bounced/unsubscribed/clicked plus downstream milestone movement (meeting-agents, agent-appointed, listed, fee) vs holdout, filterable by period and campaign. | P2 |
| R11 | Utility engine live: monthly value-refresh + local-market send for gated cohorts, stage-branched templates, batch-throttled, event-replaces-monthly rule. | P2 |
| R12 | Location normalisation for send targeting (state variants like NSW,/Victoria, placeholder suburbs like TOP3 excluded with reason BAD_LOCATION). | P2 |
| R13 | Market-event alert capability as a preview-gated manual playbook (pre-built template + audience + preview + explicit fire), not an automated trigger. | P3 |
| R14 | Property value tool events, estimates history, verified-email capture and scoped consent flow into the CRM and can trigger enrolments (§12). | P3 |
| R15 | Weekly growth report gains an email-lane block (counts, lift vs holdout, next action). | P3 |
3.2 Non-functional requirements
- N1 — Fail closed. If eligibility cannot be computed (missing data, query error), the contact is not sent to. An exclusion with reason beats an accidental send.
- N2 — At-most-once with visible failure. A unique
(enrolment_id, step_no)attempt and atomic claim prevent duplicate jobs. Ambiguous provider outcomes are surfaced for human reconciliation and never automatically retried. - N3 — Auditable. Every attempt knows its campaign/step/enrolment/address/classification; every exclusion and stop has a reason; consent evidence and campaign status changes are attributable.
- N4 — Reversible. Pause stops new claims within one tick (≤10 min); resume shows proceed/expire counts and stale rows expire rather than drain. No destructive migration of existing CRM history.
- N5 — Deliverability-safe. Marketing classification, appropriate marketing pool, batch throttles, recency warm-up, body unsubscribe and
List-Unsubscribe/List-Unsubscribe-Postheaders before the first external send. - N6 — Compliance control. Recipient-level consent evidence, sender identification and functional unsubscribe are release gates; legacy unsubscribe tags are binding. Legal/business confirmation of the recent-enquiry rescue basis gates external P1 sends, not Phase 0 engineering.
4. Current capability map and gaps (code-verified through 2026-07-11)
4.1 What already exists and is reused
| Capability | Where (verified) | Reuse in this build |
|---|---|---|
| Email send pipeline (SendGrid, IP pools, delayed dispatch) | app/Jobs/SendSimpleBodyEmail.php, app/Mail/SimpleBody.php (sets categories, ip_pool, custom_args), app/Services/Messages/SendMessageService.php | Nurture tick dispatches SendSimpleBodyEmail — no new sender is built. |
Template rendering + merge fields (@PROPERTY_SUBURB@, @CONTACT_*@, @SUBURB_*@…) | app/Services/TemplateTagsService.php, app/Models/Comms/Template.php (DB-stored HTML) | Campaign steps reference templates.id; preview renders via the same service. |
| Send + event logging | msg_emails / msg_emails_data; SendGrid webhook → msg_email_sendgrid_events via app/Services/Messages/SendgridEventWebhookService.php (signature verification option, event-hash dedupe, custom-args persistence) | Attribution backbone. Existing sends use base args and message-id fallback; wire campaign/per-email args and verify production signature enforcement. |
| Workflow engine (enrolment triggers only) | app/Services/TriggerService.php, app/Services/Workflow/Triggers/* incl. MilestoneNotChanged (hourly poller), ContactTagAdded, TagsBelongsToContact (recurring schedules) | Optional enrolment source; never the send rail (D1). |
| Batch/throttle machinery | app/Schedulers/TriggerBelongsToSend.php + PARAM_DIGEST_BATCH_AMOUNT/PERIOD (app/Services/ParametersService.php); test-send seeding in ParametersController::testDigest() | Throttle pattern copied for utility batches; testDigest pattern informs campaign test mode. |
| Milestone/downstream movement data | properties.meeting_agents_at / agent_appointed_at / listed_online_at / fee_received_at, milestones_history, denormalised property_timeline | ROI joins in CampaignReportService. |
| Staff-touch history | actions table (unified: task/note/email/sms/call ids + timestamps), tasks.status=1 for open tasks; MsgEmailObserver creates an action for every saved email | RECENT_STAFF_TOUCH excludes campaign-stamped email actions so nurture does not suppress itself; open tasks remain a review flag. |
| Unsubscribe landing flow | app/Http/Controllers/Agent/MarketingTagLinkController.php — detaches all marketing tags from contact + properties | Kept; becomes one input to eligibility rather than the whole mechanism. |
| Existing address blacklist | app/Models/Blacklist.php / singular blacklist table (type=email|phone, value) | Existing hard block input only. Do not overload it with marketing-unsubscribe provenance because it also serves lead-intake/fraud paths. |
| Active workflow email inventory | Current local snapshot: 17 active, non-deleted workflows / 57 email actions. Commercial examples: Seller Guide (61), Home Selling Masterclass NSW (107), 6 Week Home Seller Plan (109). | Re-run read-only in production in P0; classify every active action as marketing or operational and record keep/defuse decision. |
4.2 Gaps to plug
| # | Gap | Evidence (code-verified) | Fixed in |
|---|---|---|---|
| G1 | No marketing suppression gate before send. Bounce/spam/unsubscribe history is recorded but not consulted by the workflow or mass/digest send rails. | SendEmailUsingTemplate::executeAction() checks only a non-empty address; SendDigestEmail deduplicates only. The existing blacklist is not a marketing suppression ledger. | P0 |
| G2 | Intent events do not stop future marketing. unsubscribe/group_unsubscribe raise a staff notification only. spamreport marks the individual msg_emails row but does not suppress future sends. | SendgridEventWebhookService.php; event recipient email is available even when no contact/msg_email resolves. | P0 |
| G3 | Commercial legacy journeys are live without the new safety contract. The 6 Week Plan has ten staggered actions; Home Selling Masterclass mixes marketing and transactional pools. | AbstractAction::saveActionInQueue() supports delay arrays. Disabling a trigger does not cancel already-queued rows because the runner does not re-check trigger active state. | P0 |
| G4 | No campaign identity. msg_emails has only template_id; outbound custom args are just tenant_id+env (the existing per-email buildCustomArgsForEmail() helper is not even wired in). | app/Jobs/SendSimpleBodyEmail.php, app/Services/Messages/SendgridEventSupport.php. | P0 |
| G5 | No durable journey or send-attempt state. Nothing records "contact X is on campaign Y, step Z" or whether a provider call was pending, sent, failed or ambiguous. | Workflow queue rows are execution records and SendSimpleBodyEmail creates msg_emails only after Mail::send(); exceptions are logged without a send row. | P1 |
| G6 | Mass email bypasses even the weak gates. Mass-email queue rows (trigger_id −2 with settings.ip_pool) skip the subscription/allow-digest checks in sendAgentDigest(). | app/Http/Controllers/Staff/Parameters/ParametersController.php::massMessages() → app/Schedulers/TriggerBelongsToSend.php → app/Services/ContactService.php::sendAgentDigest(). | P0 |
| G7 | No audience preview. Mass email shows only a total count on confirm; no reasons, no samples, no conflicts. | massMessages() confirm step returns contacts_amount only. | P1 |
| G8 | No lane/campaign ROI reporting. Statistic pages exist but nothing joins sends → SendGrid events → milestones. | app/Services/Statistics/* has no email views. | P2 |
| G9 | No unified email-validity model. emails has no validity flag; delivery outcomes, blacklist entries, unsubscribe tags and preferences live separately. | app/Models/Email.php, blacklist, msg_email_sendgrid_events, msg_emails.status, tags and subscriptions. | P0 (live eligibility + suppression) |
| G10 | Location data is dirty for send targeting: state variants (NSW,, Victoria), placeholder suburbs (TOP3); 277 newsletter contacts unusable. | Phase 0 findings; properties.state/suburb free-text. | P2 |
| G11 | No one-click unsubscribe headers. Marketing mail has no List-Unsubscribe/List-Unsubscribe-Post payload or bare signed POST endpoint. | app/Mail/SimpleBody.php; the pinned SendGrid driver accepts arbitrary v3 payload parameters through its existing smtpapi embed channel. | P0/P1 |
| G12 | IP pool is being used as a false marketing classifier. | Commercial trigger 109 uses transactional; trigger 107 mixes pools. Purpose and transport are separate facts. | P0 |
| G13 | Nurture sends would suppress their next step as a "staff touch". | MsgEmailObserver creates an actions row and fires EmailSent/PropertyChanged whenever a MsgEmail is created. | P1 |
| G14 | No recipient-level consent evidence snapshot. | agent_subscriptions.status is a preference/schedule control; contacts.marketing_included is an agent-fee field, not consent. | P1 |
| G15 | Contact-only suppression misses duplicate records and unresolved events. | emails.email is not unique across contacts; a webhook can carry an address without resolving msg_email_id. | P0 |
5. Target architecture
EnrolmentService::enrol() — nothing writes email_enrolments directly.email_campaigns · email_campaign_steps · email_enrolments (status, current_step, next_send_at, consent evidence, outcome/stop reason, holdout)
EligibilityService reads email_suppressions, blacklist, preferences and marketing-send history; AudienceService::journeyStillValid() re-checks lane predicates
email-nurture:tick atomically creates email_send_attempts → one job claims the attempt → existing rendering/SendGrid rail
msg_emails ⇄ msg_email_sendgrid_events ⇄ property_timeline; SuppressionService writes normalized address state and stops journeys for intent suppressions
5.1 The tick loop (core runtime)
every 10 min: email-nurture:tick (withoutOverlapping, like trigger:action-queue)
for each enrolment WHERE status='active' AND next_send_at <= now() (chunked, ordered):
campaign = enrolment.campaign
if campaign.status NOT IN ('active','testing') → skip without claiming
step = campaign.steps[enrolment.current_step + 1]
if no step → status='completed'
if next_send_at is older than campaign.max_overdue_hours
→ stop EXPIRED
validity = AudienceService::journeyStillValid(enrolment)
if !validity.ok → stop PROGRESSED / FAILED_STAGE / other outcome
elig = EligibilityService::check(contact, target_address, communication_class, context)
if !elig.ok → record skip:
hard reasons (UNSUB_TAG, ADDRESS_SUPPRESSED, BLACKLISTED,
NO_EMAIL, NO_CONSENT_EVIDENCE) → status='stopped', exclusion_reason
soft reasons (FREQ_CAP, RECENT_STAFF_TOUCH) → defer next_send_at +24h (max 3 defers, then stop)
if campaign.status='testing' AND target not in test allowlist → exclude with TEST_ALLOWLIST reason
template = resolve template_id or template_map[current milestone/default]
within one DB transaction:
INSERT email_send_attempts(status='pending')
UNIQUE(enrolment_id, step_no); duplicate means another tick won → skip
advance current_step and next_send_at
after commit: dispatch job(attempt_id)
job atomically claims pending → claimed; duplicate job exits
deterministic error before Mail::send() → failed (manual retry allowed)
exception/worker death during provider call → ambiguous (NO automatic retry)
success: create MsgEmail normally; observer runs; attempt → sent + msg_email_id
stale claimed attempt >30 min → ambiguous + operator alert
respect campaign.throttle_per_hour (stop tick batch when exhausted)
Why not pre-create msg_emails: MsgEmailObserver treats creation as a real sent email, creates an activity and touches the property. Attempt state therefore lives separately; msg_emails is created only after a successful provider call. Absolute exactly-once delivery is impossible across an external provider boundary, so the deliberate contract is at-most-once with visible ambiguous outcomes.
6. Database schema
MySQL/InnoDB, Laravel timestamped migrations in database/migrations/, named indexes and repo-compatible integer widths. Use foreign keys only where the current migration/test conventions support them cleanly; the behavioural contract relies on unique/index constraints and service validation, not an assumed no-FK rule.
6.1 new email_campaigns
id bigint PK
key varchar(64) UNIQUE -- slug, e.g. 'fast-rescue-v1'; goes into custom_args
name varchar(150)
lane varchar(16) -- 'rescue' | 'utility' | 'event' | 'tool'
status varchar(16) default 'draft' -- draft|testing|active|paused|archived
description text NULL
communication_class varchar(16) default 'marketing' -- marketing|operational; nurture is marketing
ip_pool varchar(32) default 'marketing'
required_consent_basis varchar(64) -- e.g. recent_service_request
required_consent_scope varchar(64) NULL
holdout_pct tinyint unsigned default 20
throttle_per_hour int unsigned NULL -- null = param default
test_recipients json NULL -- allowlist emails used when status='testing'
settings json NULL -- touch window, max_overdue_hours (rescue default 48), cohort filters
created_by unsignedInteger NULL -- users.id
timestamps, softDeletes
INDEX idx_email_campaigns_status (status)
Status transitions: draft→testing→active⇄paused→archived. Activation fails unless consent requirements, templates, sender identity, body unsubscribe token and testing allowlist are valid. Pause takes effect within one tick. Resume confirmation shows how many overdue rows will expire and how many remain eligible; no re-anchor UI is required.
6.2 new email_campaign_steps
id bigint PK
campaign_id bigint unsigned -- email_campaigns.id
step_no smallint unsigned -- 1..n
template_id unsignedInteger NULL -- templates.id (simple case)
template_map json NULL -- stage-branched: {"lead": 123, "failed": 124, "meeting_agents": 125}
offset_hours int unsigned -- from enrolment.anchor_at (step 1) / from prev send (steps 2+)
send_window varchar(32) NULL -- e.g. '08:00-18:00' local property timezone (reuse get_time_modified pattern)
active boolean default true
timestamps
UNIQUE uq_campaign_step (campaign_id, step_no)
Exactly one of template_id/template_map must be set (validated in service). template_map implements D7 stage branching keyed on the property's current milestone label.
6.3 new email_enrolments
id bigint PK
campaign_id bigint unsigned
contact_id unsignedInteger -- contacts.id
property_id unsignedInteger default 0 -- properties.id; 0 = contact-level (avoids NULL-in-unique problem)
status varchar(16) -- active|completed|stopped|excluded|holdout
current_step smallint unsigned default 0
anchor_at datetime -- timing origin (property.created_at for rescue; enrol time otherwise)
next_send_at datetime NULL
last_send_at datetime NULL
last_msg_email_id unsignedInteger NULL
defer_count tinyint unsigned default 0
exclusion_reason varchar(64) NULL -- reason code at stop/exclude time
source varchar(16) -- workflow|manual|batch|tool
consent_basis varchar(64) -- copied at enrolment
consent_evidence_type varchar(32) -- property_enquiry|tag|pvt_event|staff_record
consent_evidence_id varchar(64) -- property id / tag id / event uuid
consent_evidence_at datetime
enrolled_by unsignedInteger NULL
timestamps
UNIQUE uq_enrolment (campaign_id, contact_id, property_id)
INDEX idx_enrolments_due (status, next_send_at)
INDEX idx_enrolments_contact (contact_id)
INDEX idx_enrolments_property (property_id)
Holdout rows are created at enrolment and never sent. P1 evidence is recent_service_request / property_enquiry / property_id / property.created_at; failure to populate it produces NO_CONSENT_EVIDENCE. PROGRESSED is a positive pre-send outcome and is not grouped with suppression/exclusion in reporting. The unique key represents one journey record per campaign/contact/property, not a lifetime ban: an explicit re-enrol may reactivate a non-intent terminal row (TEST_ALLOWLIST, EXPIRED, PROGRESSED or other non-suppression exclusion) only after eligibility, journey validity and consent evidence are re-run and the step/anchor are reset. UNSUBSCRIBED and MANUAL_STOP are never reactivated automatically.
6.4 new email_suppressions (effective address state)
id bigint PK
email varchar(255) -- lower(trim(email)); do not alter dots/plus aliases
scope varchar(32) default 'marketing'
reason varchar(32) -- unsubscribe|spam|bounce|block|invalid|dropped|manual
source varchar(32) -- sg_webhook|one_click|landing|staff|import
contact_id unsignedInteger NULL
msg_email_id unsignedInteger NULL
last_event_ref varchar(255) NULL -- sg_event_id / one-click token id / staff action id
last_event_at datetime
occurrences smallint unsigned default 1
lifted_at datetime NULL
lifted_by unsignedInteger NULL
lift_reason varchar(255) NULL
history json NULL -- compact suppression/lift evidence; append under row lock
timestamps
UNIQUE uq_email_suppression (email, scope, reason)
INDEX idx_email_suppressions_active (email, scope, lifted_at)
INDEX idx_email_suppressions_contact (contact_id)
Active means lifted_at IS NULL. Upstream SendGrid events remain the provider-event ledger; this row is effective state with compact lift evidence. Unsubscribe/spam can only be lifted with recorded re-consent. Delivery suppressions can be manually lifted after verification. Existing blacklist remains a separate hard-block input.
6.5 new email_send_attempts (outbox + failure state)
id bigint PK
campaign_id bigint unsigned
enrolment_id bigint unsigned
step_no smallint unsigned
contact_id unsignedInteger
email varchar(255) -- exact normalized target selected at claim
communication_class varchar(16) default 'marketing'
status varchar(16) -- pending|claimed|sent|failed|ambiguous
msg_email_id unsignedInteger NULL
error text NULL
claimed_at datetime NULL
sent_at datetime NULL
timestamps
UNIQUE uq_attempt_step (enrolment_id, step_no)
INDEX idx_attempts_status_age (status, created_at)
The unique row is the idempotency guard. A failed attempt can be deliberately reset/requeued by an admin command after eligibility and journey validity are re-checked. An ambiguous attempt cannot be retried until a human checks SendGrid activity and records the reconciliation decision.
6.6 change msg_emails — additive columns
ALTER TABLE msg_emails
ADD campaign_id bigint unsigned NULL,
ADD enrolment_id bigint unsigned NULL,
ADD step_no smallint unsigned NULL,
ADD communication_class varchar(16) NULL,
ADD ip_pool varchar(32) NULL,
ADD INDEX idx_msg_emails_campaign (campaign_id, created_at);
Written after a successful send. Legacy call sites default to operational until their P0 inventory classification is explicit; unclassified calls log loudly and P0 cannot exit while an active workflow action remains unclassified. Frequency caps count communication_class='marketing', not IP pool.
6.7 change Suppression and unsubscribe lifecycle
SuppressionService::suppressAddress() is the only writer used by SendGrid webhook events, the signed one-click POST endpoint, the existing body preference flow and staff/manual actions. It upserts normalized address state even when contact resolution fails. For unsubscribe/spam it also applies the canonical unsubscribed tag (matched by name across duplicate tag rows) and, once P1 tables exist, stops all active enrolments for the resolved contact synchronously. The one-click URL is HMAC signed, login-free, idempotent, CSRF-exempt only for that route, rate-limited, returns a generic response and remains valid for at least 30 days. GET shows a confirmation/preferences page and never unsubscribes automatically.
6.8 P3 Property value tool tables (in CRM)
pvt_events
id bigint PK · uuid char(36) UNIQUE · occurred_at datetime
event_type varchar(32) -- value_requested|value_viewed|value_refreshed|return_visit|
-- cost_to_sell_used|report_requested|report_unlocked|
-- email_captured|email_verified|consent_given
session_id varchar(64) NULL
address_full varchar(255) NULL · suburb varchar(100) NULL · state char(3) NULL · postcode varchar(8) NULL
property_type varchar(20) NULL · beds tinyint NULL
estimate_low/mid/high decimal(12,2) NULL · estimate_source varchar(32) NULL · confidence varchar(16) NULL
email varchar(255) NULL · consent json NULL -- {scope, wording_version, ts, source_url}
contact_id unsignedInteger NULL · property_id unsignedInteger NULL -- resolved async
payload json NULL · timestamps
INDEX (event_type, occurred_at) · INDEX (contact_id) · INDEX (property_id)
property_estimates -- history, so "your estimate changed" emails are honest
id bigint PK · property_id unsignedInteger · as_of date
low/mid/high decimal(12,2) · source varchar(32) -- rea_cache|yipm|oth_suburb_type|pvt
confidence varchar(16) NULL · sales_support smallint NULL -- recent sales count behind the number
timestamps · UNIQUE (property_id, as_of, source)
7. Service layer and commands
Per repo rules: thin controllers, business logic in app/Services/. New namespace App\Services\EmailNurture\. All new code paths log through the standard Laravel log with a [email-nurture] prefix.
| Component | Contract |
|---|---|
EligibilityServiceapp/Services/EmailNurture/EligibilityService.php |
check(Contact $c, string $targetEmail, string $communicationClass, array $ctx=[]): EligibilityResult returns ok, reasons, flags and normalized target. Marketing checks live: syntax/email ownership, email_suppressions, singular blacklist, all three legacy unsubscribe tag names (regardless of tags.marketing), subscription disabled, 2/14-day marketing count, and rescue-only human staff touch. Campaign-stamped email actions are excluded from staff-touch. checkBulk() uses set-based queries; gateOrAbort() runs again inside marketing send paths. Operational class is not blocked by marketing preferences. |
SuppressionService…/SuppressionService.php |
Normalizes with lower(trim(email)); writes/lifts effective suppression; applies canonical unsubscribe tag and synchronously stops contact journeys for intent suppression; records unresolved event addresses; exposes idempotent one-click/body/staff/webhook entry points. It never uses Gmail dot/plus folding and never conflates the existing blacklist with marketing preference. |
EnrolmentService…/EnrolmentService.php |
enrol(Campaign, Contact, ?Property, ConsentEvidence, string $source, bool $execute=false, bool $reactivate=false) runs eligibility and cohort validity, snapshots consent evidence, assigns deterministic holdout, computes timing and dedupes on the unique key. Missing required evidence fails closed. Explicit $reactivate follows the §6.3 terminal-row rules and records the prior reason, actor and timestamp in the audit log; intent stops cannot be reactivated by this path. stop(), stopAllForContact() and stopAllForCampaign() are idempotent. Dry-run is the default. |
AudienceService…/AudienceService.php |
One predicate family per cohort: rescueCohort(), valueRefreshCohort(), localUtilityCohort(). Preview/enrol use the query; tick uses journeyStillValid(enrolment) built from the same predicates so progressed, failed/closed and stale leads never receive the next step. Includes rescueTouchOverlap(). Cohort-only outcomes (stage/location/duplicate) do not pollute marketability reason codes. |
PreviewService…/PreviewService.php |
preview(Campaign): PreviewResult — candidate count, would-enrol count, holdout count, excluded-by-reason breakdown, flag counts (open-task, recent-touch), 20 sample rows (name/suburb/stage/last-touch/eligibility), data-freshness stamp. Serialisable for the UI data endpoint and printable by the console command's renderPreview() (pattern: app/Console/Commands/GoogleAdsImportAuctionInsightsCommand.php). |
NurtureTickService…/NurtureTickService.php |
Implements §5.1. Every 10 minutes with withoutOverlapping; still uses a database unique claim because scheduler locks are not correctness. Re-checks expiry, journey validity and eligibility; atomically creates attempt + advances state; dispatches after commit; marks stale claimed attempts ambiguous; respects per-campaign throttle. |
SendAttemptService…/SendAttemptService.php |
Claims one pending attempt atomically, renders and sends, then creates MsgEmail normally and marks sent. Pre-provider failures become failed. Provider-call exception/worker death becomes ambiguous and alerts. Retry command accepts failed only; ambiguous requires a recorded manual reconciliation first. |
CampaignReportService…/CampaignReportService.php |
Per campaign/period: enrolled, sent, delivered, bounced, unsubscribed, clicked (from msg_email_sendgrid_events by msg_email_id); downstream movement = properties whose meeting_agents_at/agent_appointed_at/listed_online_at/fee_received_at (via property_timeline) turned non-null after first send; identical stats for the holdout cohort (movement after their would-have-sent time); lift = sent-cohort rate − holdout rate with cohort sizes shown. No opens anywhere in the headline (MPP). |
7.1 Console commands
| Command | Behaviour |
|---|---|
email-nurture:audit-workflows | Read-only inventory of every active, non-deleted workflow email action, template, pool and explicit communication classification. P0 exit requires zero unclassified active actions. |
email-nurture:backfill-suppressions {--execute} | Dry-run by default. Seeds effective address suppression from legacy unsubscribe tags, SendGrid events/statuses and existing blacklist comparison; reports source counts and duplicates before execute. |
email-nurture:preview {campaign-key} {--limit=20} | Read-only. Renders PreviewService output as console tables. Safe on production data. |
email-nurture:enrol {campaign-key} {--execute} | Default dry-run (prints what would enrol, by reason). --execute writes enrolments. Mirrors RollbackContactsRonasSyncCommand convention. |
email-nurture:tick {--once} | The send loop. --once processes a single due enrolment (debugging). |
email-nurture:retry-attempt {attempt-id} {--execute} | Failed attempts only; dry-run shows current eligibility/journey checks. Ambiguous attempts are rejected until manually reconciled. |
email-nurture:report {campaign-key} {--period=30d} | Console version of the performance report; feeds the weekly readout until the UI page exists. |
7.2 Touch points in existing code modifications
- app/Jobs/SendSimpleBodyEmail.php — accept attempt/campaign/classification context; do not pre-create
MsgEmail; merge campaign args and stamp successful send rows. Refactor only enough to surface pre-provider vs provider-call failure toSendAttemptService. - app/Services/Messages/SendgridEventWebhookService.php — call
SuppressionServicefor unsubscribe/group-unsubscribe/spam/bounce/block/invalid/dropped as defined; unresolved addresses still persist. - app/Mail/SimpleBody.php — include body unsubscribe token and SendGrid v3
List-Unsubscribe/List-Unsubscribe-Postheaders on marketing class; add signed POST route/controller. - app/Services/Workflow/Actions/SendEmailUsingTemplate.php (+
…ToContact) — pass explicit communication class; marketing-class sends callgateOrAbort(). Pool never decides class. - app/Services/ContactService.php::sendAgentDigest() / app/Jobs/SendDigestEmail.php — vendor digest and mass path are marketing and gate at send time. Agent digest remains operational for P0 by explicit decision, not by contact-type loophole.
- app/Http/Controllers/Staff/Parameters/ParametersController.php::massMessages() — confirm step returns the PreviewService reason breakdown instead of a bare count (small change, big safety win).
- app/Observers/MsgEmailObserver.php is not disabled; instead, staff-touch queries exclude actions joined to
msg_emails.campaign_id IS NOT NULL. Audit activeEmailSentworkflows before the first test send. - routes/web.php / HTTP kernel — register and attach
IsAdminStaffto campaign administration; all staff may use per-contact stop. Add sharedasStaff()test helper if it does not exist on the target branch.
8. CRM UI surfaces
Stack: Blade + jQuery on layouts.base, statistic pages via the AbstractStatisticController two-route pattern (@plot GET view + @data POST JSON). No Vue needed for P1–P2 — plain tables are the fastest route to trustworthy. Admin-gated via IsAdminStaff middleware initially.
8.1 essential Campaign list + detail — /email-nurture
New app/Http/Controllers/Staff/EmailNurture/CampaignController.php, views under resources/views/staff/email-nurture/.
EMAIL NURTURE — CAMPAIGNS [+ New campaign] ┌────────────────────┬────────┬─────────┬─────────┬──────────┬──────────┬─────────────┐ │ Campaign │ Lane │ Status │ Active │ Next 24h │ Sent 30d │ Actions │ ├────────────────────┼────────┼─────────┼─────────┼──────────┼──────────┼─────────────┤ │ fast-rescue-v1 │ rescue │ ACTIVE │ 41 │ 12 │ 214 │ Pause View │ │ value-refresh-v1 │ utility│ TESTING │ 6* │ 6 │ 18 │ Stop View │ │ vic-market-jul │ utility│ DRAFT │ — │ — │ — │ Preview │ └────────────────────┴────────┴─────────┴─────────┴──────────┴──────────┴─────────────┘ * testing = internal allowlist only Global cap: 2 sends / 14 days
Detail page shows: communication class, consent requirement, sender identity, steps/templates with rendered preview, offsets, overdue limit, throttle, holdout and test allowlist; live audience preview; enrolments; and send attempts grouped into pending/claimed/sent/failed/ambiguous. Pause/Stop-all require confirmation. Resume confirmation shows rows that will proceed versus expire. Ambiguous attempts link to a manual reconciliation action and can never be blindly retried.
8.2 essential Audience preview (embedded in campaign detail + standalone)
AUDIENCE PREVIEW — fast-rescue-v1 computed 2026-07-02 09:41 [Refresh]
Candidates matching cohort ................ 399
├─ would enrol (send) ................... 104 ├─ holdout (20%) ............ 26
└─ excluded ............................. 269
NO_EMAIL ........................... 262 RECENT_STAFF_TOUCH (14d) .... 74
UNSUB_TAG .......................... 0 FREQ_CAP .................... 3
ADDRESS_SUPPRESSED ................. 11 (overlaps shown per-contact)
Flags (would send, but review): OPEN_TASK_CONFLICT 38
┌─ Sample (20) ────────────────────────────────────────────────────────────────────┐
│ Contact Suburb Stage Last staff touch Eligibility │
│ J. Smith Tarneit Report Delivered 11d ago (call) OK ⚑ open task │
│ K. Nguyen Baldivis Email Nurture — OK │
│ … │
└──────────────────────────────────────────────────────────────────────────────────┘
This is R7. It renders PreviewService output — the same object the enrol command prints. The rescue view must include the touch-overlap numbers so the 137×239 question is answered on screen, permanently.
8.3 essential Email journey card on contact + property screens
New partial resources/views/staff/partials/email-journey/card.blade.php, included in staff/contact/show.blade.php and staff/property/show.blade.php following the existing @include card pattern (e.g. staff/partials/activities/card), backed by a controller partial-refresh endpoint like #other_contacts_table.
✉ EMAIL JOURNEY Marketable: YES fast-rescue-v1 step 1/2 sent 30 Jun 09:12 (delivered, clicked) next: step 2 · 2 Jul 09:12 value-refresh-v1 EXCLUDED — RECENT_STAFF_TOUCH (call 28 Jun) Last marketing sends: 1 in last 14 days (cap 2) [Stop all nurture for this contact]
This card is also the staff-conflict safeguard: consultants see what marketing sent before they dial, and can stop all nurture for a contact in one click. It also shows address suppression and failed/ambiguous attempt warnings. Campaign-generated email activities remain visible here but are excluded from RECENT_STAFF_TOUCH calculations.
8.4 essential (P2) Lane performance report — statistic pattern
New app/Http/Controllers/Staff/Statistic/EmailNurtureReportController.php extending AbstractStatisticController (routes stat.email-nurture + stat.data.email-nurture), view extends staff.statistic.main with the period filter partial.
EMAIL LANE PERFORMANCE — 1 Jun → 30 Jun ┌───────────────┬───────┬───────┬────────┬───────┬────────┬───────────────┬──────────────┐ │ Campaign │ Sent │ Deliv │ Bounce │ Unsub │ Click │ Meeting-agents│ vs holdout │ ├───────────────┼───────┼───────┼────────┼───────┼────────┼───────────────┼──────────────┤ │ fast-rescue-v1│ 214 │ 208 │ 4 │ 1 │ 31 │ 11/214 (5.1%) │ +2.3pp (n=54)│ │ value-refr-v1 │ 480 │ 471 │ 6 │ 3 │ 74 │ 6/480 (1.3%) │ +0.6pp (n=120)│ └───────────────┴───────┴───────┴────────┴───────┴────────┴───────────────┴──────────────┘ Downstream (fee received) is lagged — treat <60-day windows as provisional.
8.5 nice-to-have (defer)
- Email Journey Dashboard (graphical, cross-campaign live view) — the campaign list covers the need numerically first.
- Contact timeline visualisation merging sends/opens/touches — the journey card + existing activities card cover it.
- Vue campaign builder — Blade forms are fine at <10 campaigns.
- Weekly growth report email block — P3, automate only once numbers are trusted.
- Per-suburb drill-downs on the performance page.
9. Testing strategy
Repo rules are strict and this spec follows them: use DatabaseTransactions (shared test DB — never RefreshDatabase) and $this->createMock() (never Mockery). Most existing tests hand-seed rows; do the same (factories are minimal in this repo). For staff feature tests, add one shared asStaff() helper if it is still absent on the implementation branch, then use it consistently.
9.1 Unit tests (tests/Unit/Services/EmailNurture/)
| Test class | Must cover |
|---|---|
EligibilityServiceTest | NO_EMAIL/invalid syntax; existing blacklist; all three unsubscribe names even with marketing=0; disabled subscription; each active address-suppression reason; case normalization; two contacts sharing one address; 2 marketing sends inside 14d; human action at 13d vs 15d; campaign email action excluded from staff-touch; OPEN_TASK_CONFLICT flag; fail closed on query error; operational class not blocked by marketing preference. |
SuppressionServiceTest | Webhook with resolved contact and unresolved address; unsubscribe/spam escalates to contact tag + journey stop; bounce/block/invalid/dropped address-only; duplicate event idempotent; lift rules/evidence; lower/trim normalization; two contacts sharing one address both unmarketable; one-click repeated POST idempotent. |
EnrolmentServiceTest | Holdout determinism/distribution; dedupe; consent evidence snapshot; missing required evidence ⇒ excluded NO_CONSENT_EVIDENCE; dry-run writes nothing; anchor/send-window arithmetic; stop-all idempotent; explicit re-enrol of TEST_ALLOWLIST/EXPIRED/PROGRESSED re-runs all gates and resets timing; UNSUBSCRIBED/MANUAL_STOP reactivation is rejected. |
NurtureTickServiceTest | Due row creates one attempt and advances; two tick passes create exactly one attempt/dispatch; paused untouched; test allowlist enforced; progressed/failed-stage/expired stop reasons; soft defer/3 strikes; default template-map fallback; throttle; last step completed; campaign-generated action does not suppress next step. |
SendAttemptServiceTest | Two job executions claim once and send once; deterministic pre-provider failure ⇒ failed; exception during provider call ⇒ ambiguous/no retry; stale claimed ⇒ ambiguous; successful send creates one MsgEmail and stamps classification/campaign args; failed retry re-checks eligibility; ambiguous retry rejected pending reconciliation. |
CampaignReportServiceTest | Seeded fixture: 3 sent contacts (1 converts post-send), 2 holdout (0 convert) ⇒ correct rates and lift; conversions before first send are not counted; bounce/unsub counted from seeded sendgrid events. |
AudienceServiceTest | Rescue cohort boundary cases: property 23h old (out), 25h (in), 8d (out); meeting_agents_at set (out); touch-overlap join returns correct intersection on seeded data. |
9.2 Feature tests (tests/Feature/Staff/EmailNurture/)
CampaignControllerTest— registered admin middleware, list/detail for admin, 403 for non-admin; activation lint rejects missing consent requirement, sender identity, body-unsubscribe token or internal allowlist; pause attributed; resume shows proceed/expire counts; staff stop-all works.PreviewEndpointTest— JSON shape (counts, reasons, samples) matches PreviewService on seeded data; no PII beyond what staff already see.WebhookSuppressionTest— signed unsubscribe/spam/delivery events write effective address state; unsigned event rejected when verification enabled; unresolved address still suppresses; event-hash dedupe unchanged.OneClickUnsubscribeTest— signed bare POST requires no login/PII, is CSRF-exempt only for the route, idempotently suppresses the exact target address and stops active contact journeys; GET never suppresses.SendSimpleBodyEmailCampaignArgsTest— successful attempt stamps msg_emails and provider payload with campaign/enrolment/attempt/classification/msg_email_id; marketing payload has body link plus both unsubscribe headers.LegacySendGateTest— marketing-class workflow/mass/vendor digest re-checks suppression at send time; operational class unaffected; pool value alone cannot bypass or activate the gate.WorkflowClassificationTest— inventory command ignores soft-deleted triggers/actions, reports every active email action and exits non-zero/blocks P0 completion when any is unclassified.
9.3 E2E (e2e-tests/tests/crm/, Playwright)
email-nurture-preview.spec— open campaign detail, refresh preview, assert counts render and reason table is non-empty.email-journey-card.spec— contact with seeded enrolment shows the card; "Stop all nurture" stops it.
9.4 Manual QA checklist (before any real cohort)
- Test-mode send to internal allowlist: merge tags resolve, sender/reply-to and communication class are correct, body unsubscribe present, both headers arrive as real headers, custom args visible, provider message id reconciles.
- Unsubscribe via body and one-click header ⇒ address suppression and journey stop are visible immediately; contact excluded from the next preview. GET link scanner request does not unsubscribe.
- Pause campaign mid-flight ⇒ no sends on next two ticks.
- Resume after overdue threshold ⇒ confirmation shows proceed/expire counts and stale rescue rows do not send.
- Force deterministic send failure and ambiguous provider exception ⇒ both visible in campaign detail/report; only failed is retryable.
- Preview counts vs
email-nurture:enroldry-run counts are identical (same code path proof).
10. Phased delivery plan
Phase 0 Safety substrate — code complete; production reconciliation pending
| Deliverable | Maps to |
|---|---|
| Run read-only production workflow inventory. Classify every active, non-deleted email action as marketing or operational; record keep/defuse decision. For defused journeys, disable the trigger and pending queue rows using the Stage, Validate, Batch-Write pattern. | R3–R4 / G3, G12 |
email_suppressions + SuppressionService + dry-run backfill from legacy tags, SendGrid events/statuses and blacklist comparison. Reconcile counts before execute. | R1–R2 / G1, G2, G9, G15 |
| Signed one-click POST and body unsubscribe converge on SuppressionService; normalized address writes work even with no resolved contact. Add List-Unsubscribe payload support and verify its generated payload in automated tests; defer the single real internal delivery check to P1. | R2, R9 / G11 |
| Explicit communication class and send-time gate in classified workflow, mass and vendor-digest paths; mass-email confirmation shows reason breakdown. Agent digest remains explicitly operational for P0. | R3 / G6, G7, G12 |
msg_emails classification/campaign columns + custom-args wiring. Do not create campaign rows or send attempts yet. | D3 / G4 |
Exit criteria: zero active workflow email actions unclassified; keep/defuse register approved; defused triggers inactive and pending queue count zero; suppression backfill reconciled with explained differences; duplicate-contact and unresolved-webhook tests green; signed one-click/body unsubscribe suppresses the normalized address; unsigned webhook rejected when verification is enabled; queued mass email suppressed after enqueue is skipped at send time; no new nurture email is sent.
Phase 1 Rescue pilot — operating layer code complete; live pilot not started
| Deliverable | Maps to |
|---|---|
Migrations: email_campaigns, email_campaign_steps, email_enrolments, email_send_attempts. Services: Enrolment, Audience/journey guard, Preview, NurtureTick and SendAttempt. Commands: preview / enrol / tick / retry-attempt / report. | R5–R8 / G5, G13, G14 |
| Campaign list + detail + embedded audience preview UI (admin-gated). | R5, R7 / G7 |
| Email journey card on contact + property show pages, with per-contact stop. | R6 |
fast-rescue-v1: 2 marketing-class steps (+24h, +72h, send window 08:00–18:00 property-local), 48h overdue expiry, recent-enquiry consent evidence, 2 trust-building templates, 20% holdout, internal testing → 15-lead micro-cohort → steady flow. | R8, R9 |
Exit criteria: attempt race tests prove one dispatch; failure/ambiguous states are visible and obey retry rules; journey guard stops progressed/stale leads; campaign-created action does not block step 2; activation lint passes; internal Gmail/Outlook/iCloud test proves headers/body unsubscribe; legal/business sign-off records rescue consent basis and recency; 15-lead lane completes with zero suppressed sends/collisions. The micro-cohort validates operation, not statistical lift.
Phase 2 Utility engine — ~1.5–2 dev weeks
| Deliverable | Maps to |
|---|---|
| Location normalisation pass using a static state-variant map and placeholder blocklist inside AudienceService. Add a mapping table only if real exceptions outgrow this. | R12 / G10 |
value-refresh-v1: monthly, stage-branched via template_map (failed vs meeting-agents vs appointed copy), freshest cohort first (value data 30–180d old), batch enrolment command + throttle. | R11 / D7 |
local-market-vic-v1: monthly VIC top-suburb clusters (Melbourne, Tarneit, Point Cook, Werribee…), suburb merge fields from existing @SUBURB_*@ stats; skip-if-stale rule (no send when suburb data hasn't refreshed). | R11 |
| Lane performance statistic page (holdout lift columns). | R10 / G8 |
| Legacy utility consent-basis decision and evidence mapping are a P2 entry gate; no inferred backfill is invented in code. | D11 / G14 |
Exit criteria: one full monthly utility cycle for VIC with bounce rate <2%, complaint rate <0.1%, zero suppressed sends; performance page reconciles with SendGrid dashboard counts (±2%); event-replaces-monthly logic proven by frequency-cap test.
Phase 3 Scale, events, property value tool — ~2–3 dev weeks, partly parallel with tool build
- Utility expansion NSW/QLD; recency-cohort warm-up continues.
- Market-event playbook: pre-built campaign (status draft), rate-decision template, audience by state, preview-gated manual activation (R13).
- PVT ingestion endpoint +
pvt_events/property_estimates+ resolver job + enrolment triggers (R14, §12). - Weekly growth report email block (R15); dormant-lead expansion decisions data-driven from P2 lift numbers.
11. Dogfooding and rollout protocol
- Read-only first. Every cohort/preview command is dry-run by default; production dry-runs are safe and encouraged (matches the workspace production-write safety rule).
- Internal allowlist. Campaign in
testingsends only to seeded internal contacts (Thomas, Felix, a Gmail, an Outlook, an iCloud address — deliverability spread). Run the full manual QA checklist (§9.4). - Micro-cohort. Rescue: the current-open 1–7d lane (~15 leads). Watch for one week: delivery, failures/ambiguous attempts, replies, staff feedback, zero conflicts. Do not claim causal lift from this sample.
- Steady state with holdout. Rescue monthly flow (~104 send / 26 holdout at current volumes). Utility: freshest 500 by recency, then expand by cohort while bounce <2% and complaints <0.1%.
- Weekly readout ritual.
email-nurture:reportnumbers into the weekly growth review (Telegram/HTML per existing delivery rails). Every campaign gets a keep / fix / kill call monthly. Downstream fee data lags — hold ROI judgements on any lane younger than 60 days. - Kill criteria (pre-agreed). Pause a campaign immediately if: complaint rate >0.2%, bounce >5% on a batch, unsub >2% on a send, or consultants report ≥3 collision incidents in a week.
12. Property value tool instrumentation contract
The tool is being built separately; this section is the contract it must satisfy so email triggers work later. Valuation sources already investigated: REA cache / YIPM / OnTheHouse suburb-type medians (docs/odin-onthehouse-valuation-fallback-investigation-2026-06-22.md).
12.1 Events the tool must emit (→ pvt_events, §6.8)
| Event | When | Email use |
|---|---|---|
value_requested / value_viewed | Address submitted / estimate shown | Rescue enrolment for non-progressors; baseline for refresh triggers |
email_captured / email_verified | Contact step; magic-link "save your estimate" click | The capture engine. Verified = valid email by construction — directly attacks the 66% no-email problem |
consent_given | Explicit checkbox, scoped: my_property | local_market | both, with wording_version + ts + source_url | Legal basis per lane; "my property" converts higher and is the more valuable trigger |
value_refreshed | Recomputed estimate, with delta | Value-refresh email when |delta| ≥ threshold (start: 3% or $25k) |
return_visit | Known contact revisits value page | High-intent reactivation signal → rescue-style enrolment |
cost_to_sell_used / report_requested / report_unlocked | Calculator/report interactions | Commission-clarity module targeting; strongest-intent cohort |
12.2 Rules
- Value first, contact second (LocalAgentFinder-validated sequencing), but email verification is the WREA upgrade: the estimate is shown free; saving/tracking it requires the verified email.
- Normalise location at capture (canonical state/suburb/postcode) — prevents recreating the G10 mess.
- Estimates are history, not a scalar (
property_estimates): "your estimate changed" emails must reference real prior values, withsales_supportrecorded so alerts can be confidence-gated (a suburb "median" backed by zero recent sales — e.g. the Moore Park case — must not trigger a "market moved" email). - Delivery to CRM: authenticated POST
/api/pvt/events(batchable, uuid-idempotent). An async resolver job matches address→property and email→contact, backfillingcontact_id/property_id; unmatched events keep raw fields for later resolution. - Enrolment wiring (P3): resolver outcomes call
EnrolmentService(e.g. verified+consented+no meeting-agents within 24h ⇒ rescue; consent scope my_property ⇒ value-refresh watch). No direct sends from the tool, ever — everything flows through the same eligibility/journey rails.
13. Risks and safeguards
| Risk | Safeguard (where in this spec) |
|---|---|
| Existing commercial workflow sends without the safety contract | P0 production inventory, explicit classification and keep/defuse decision; disable pending rows as well as triggers (D5, R3–R4) |
| Send to unsubscribed/bounced address (incl. duplicate contacts) | Normalized address suppression + blacklist/tags/preferences; unresolved webhooks persist; send-time gate; intent suppression stops contact journeys (R1–R3) |
| Over-messaging across overlapping audiences | Global 14-day/2-send cap in eligibility; event-replaces-monthly (D6); per-campaign throttle |
| Marketing emails collide with staff, or step 1 suppresses step 2 | RECENT_STAFF_TOUCH uses human actions only; campaign email actions excluded; OPEN_TASK_CONFLICT flag, journey card and staff stop control (§7–§9) |
| Misleading ROI (MPP-inflated opens; rescue claiming staff-won conversions) | No opens in headline metrics; mandatory deterministic holdout; lift-vs-holdout as the only success metric; lagged-fee warning on the report (D4, R10) |
| Deliverability damage from cold 730-day list | Marketing IP pool isolation (existing), recency-cohort warm-up, batch throttles, List-Unsubscribe headers (G11), kill criteria thresholds (§11) |
| Consent/unsubscribe/sender evidence incomplete | Per-enrolment evidence snapshot; activation lint; named external-send sign-off; sender footer; body + one-click unsubscribe; synchronous stop (N6, §14) |
| Stale/thin market data in sends | Skip-if-stale rule on utility sends; confidence gates (sales_support) on event/refresh triggers (§10 P2, §12) |
| Double-send or silent loss on crash/provider uncertainty | Unique send attempt, atomic claim, explicit failed/ambiguous state, stale-claim sweep and no auto-retry for ambiguous outcomes (D10, N2) |
| Paused or delayed campaign drains stale rescue emails | Journey validity before every send, 48h overdue expiry, resume proceed/expire count (D12) |
| Preview drift ("preview said 104, it sent 260") | AudienceService is the single home of cohort SQL; preview = enrol dry-run by construction; §9.4 check makes it a release gate |
14. Locked decisions and release gates
There are no architecture questions left for Felix to decide. The following choices are locked; the checks below validate external state without reopening the design.
- SMS: excluded from P1. Revisit as
fast-rescue-v2only after the email pilot readout and separate SMS consent/cost review. - Frequency: maximum 2 marketing-class emails per normalized address/contact in a rolling 14 days. Configurable after evidence; events replace rather than stack.
- Permissions: campaign administration/pause/resume is admin-only; all staff may stop nurture for a contact. Register and test
IsAdminStaff. - Unsubscribe ownership: CRM is source of truth. Use body unsubscribe plus List-Unsubscribe/List-Unsubscribe-Post to the signed CRM endpoint; no SendGrid ASM dependency. Validate one real internal header delivery before external sends.
- PVT integration: authenticated, UUID-idempotent POST to CRM; no shared database and no direct sends from the tool.
- Templates and sender: Thomas owns the two P1 rescue templates and chooses visible From name/reply identity. Activation is blocked until body unsubscribe and accurate WREA business/contact footer are present.
- Consent release gate: before the first external rescue send, record counsel/business confirmation that a recent agent-comparison enquiry supports the two rescue emails and choose the recency limit (implementation default: 7 days). P2 separately requires a decision on legacy newsletter consent evidence.
- Production/runtime validation: re-run workflow inventory and pending queue counts read-only; verify SendGrid webhook signature enforcement/key, current unsubscribe-tag counts, EmailSent trigger interactions and real header delivery. Differences are documented before any write/send.
- Branch/test validation: implement against current
pre_production, not the feature branch used for the July 11 review. CI is the suite-health authority; useQUEUE_DRIVER=syncand add a sharedasStaff()helper if absent.
15. Appendix
15.1 Key code references (reviewed through 2026-07-11; re-check on implementation branch)
| Thing | Where |
|---|---|
| Send job / Mailable (custom args, IP pool, categories) | app/Jobs/SendSimpleBodyEmail.php · app/Mail/SimpleBody.php |
| Custom args helpers (base + unwired per-email) | app/Services/Messages/SendgridEventSupport.php |
| SendGrid webhook (signature, dedupe, custom-args round-trip) | app/Services/Messages/SendgridEventWebhookService.php · route POST sendgrid-events |
| SendGrid transport and headers | vendor/s-ichikawa/laravel-sendgrid-driver/src/Transport/SendgridTransport.php (pinned dependency; do not edit vendor) · application header payload via app/Mail/SimpleBody.php sendgrid/x-smtpapi embed data |
| Email/event tables | msg_emails + msg_emails_data · msg_email_sendgrid_events (migration 2026_02_25_120000) |
| Email-to-action observer | app/Observers/MsgEmailObserver.php — campaign-stamped sends must not count as human staff touches |
| Merge fields | app/Services/TemplateTagsService.php (@FIELD@ tokens incl. @SUBURB_*@, @CONTACT_MARKETING_TAG_LINK@) |
| Workflow engine (enrolment triggers) | app/Services/TriggerService.php · app/Services/Workflow/Triggers/* · queue table triggers_actions_queue · runner trigger:action-queue (app/Commands/RunActionQueueCommand.php) |
| Mass email / digest rail + throttle params | ParametersController::massMessages() · app/Schedulers/TriggerBelongsToSend.php · app/Services/ContactService.php::sendAgentDigest() · app/Jobs/SendDigestEmail.php |
| Unsubscribe landing | app/Http/Controllers/Agent/MarketingTagLinkController.php (detaches marketing tags contact+properties) |
| Existing hard-block input | blacklist table — retain as a separate eligibility input; do not silently merge its meaning into consent or preference state |
| Milestones / downstream movement | properties.*_at stage columns · milestones_history · property_timeline |
| Staff touches / tasks | actions (unified log) · tasks (STATUS_OPEN=1) |
| Tags | tags (marketing bool) + entity_tags pivot |
| Statistic page pattern to copy | app/Http/Controllers/Staff/Statistic/FinishedTasksReportController.php (+ AbstractStatisticController, view staff.statistic.main) |
| Dry-run command pattern to copy | app/Console/Commands/GoogleAdsImportAuctionInsightsCommand.php (--dry-run + renderPreview()) |
| Test conventions | DatabaseTransactions, createMock(), hand-seeded rows; add/reuse a shared asStaff() helper on the implementation branch; Playwright in e2e-tests/ |
15.2 Phase 0 lane sizing (2026-06-30 dry run, read-only)
| Lane | Properties | Contacts | Note |
|---|---|---|---|
| Fast rescue — current open 1–7d | 15 | 15 | Pilot lane |
| Fast rescue — 30d 24h-no-progress flow | 137 | 136 | Gross 399; 262 no email; 239 recent-touch (overlap unquantified → R7) |
| Local market utility | 10,935 | 10,083 | Largest durable lane |
| Market event alert | 10,958 | 10,104 | Same audience as utility ⇒ merged system (D6) |
| Property-value refresh | 6,312 | 5,975 | Gross 18,365; 11,984 no email |
| Explicit tool tags | 14 | 10 | Too small — hence §12 |
Suppression signals: MailChimp:unsubscribed 2,132 · unsubscribed 486 · Unsubscribe to Newsletter 11 · SendGrid 730d: dropped 446 ev/250 contacts, bounce 315/216, spamreport 4/4. State pools (gated): VIC 3,115 · NSW 2,554 · QLD 2,052 · WA 1,360 · SA 524 · TAS 254 · ACT 252 contacts.
15.3 Phase 0 workflow inventory baseline (local read-only review, 2026-07-11)
| Inventory item | Observed baseline | Required production action |
|---|---|---|
| Active, non-deleted email workflows | 17 workflows / 57 actions | Re-run email-nurture:audit-workflows --production --dry-run; record counts and differences before any write |
| Commercial workflows requiring classification | 61 Seller Guide; 107 Home Selling Masterclass NSW; 109 6 Week Home Seller Plan | Assign communication_class and an explicit keep/defuse decision to each |
| 6 Week Home Seller Plan | Ten staggered actions over six weeks | Defuse trigger execution and disable any pending queue rows before the pilot |
| Soft-deleted local rows | 110 and 112 had active flags but were soft-deleted | Exclude deleted workflows from active counts; report them separately for audit clarity |
| Pending queue rows for reviewed workflows | None in the local snapshot | Do not assume production is empty; count and sample pending rows read-only immediately before defusing |
This is a baseline, not production proof. Phase 0 deliberately repeats the inventory against production before changing workflow or queue state.
15.4 Related documents
- artifacts/wrea/email-nurture-strategy/2026-07-02/claude-fable-context-brief.md — context brief this spec answers
- reports/wrea-email-nurture-strategy-crm-capability-2026-06-30.html — strategy & CRM capability review
- artifacts/wrea/email-nurture-phase0/2026-06-30/phase0-findings.md — dry-run numbers
- artifacts/wrea/competitor-reviews/local-agent-finder-2026-06-26/…/report/index.html — LAF valuation-tool teardown (capture sequencing)
- docs/odin-onthehouse-valuation-fallback-investigation-2026-06-22.md — valuation data sources for §12
Original specification authored by Claude (Fable 5) on 2026-07-02. Version 2.0 was jointly reviewed and corrected by Codex and Claude Code on 2026-07-11 against the strategy evidence and a read-only inspection of the wrea-multi-project/wrea codebase. Reconcile code references and production inventory at implementation start.