Email Pipeline

MoMail processes emails through a sophisticated pipeline that ensures reliable delivery, vector indexing, and real-time notifications.

Pipeline Overview

graph LR
    A[Incoming Email] --> B[Email Worker]
    B --> C[Queue Handler]
    C --> D[Database Storage]
    C --> E[Vector Embedding]
    C --> F[Webhook Notification]

Step-by-Step Flow

1. Email Reception

When an email is sent to your domain:

1. The sender’s mail server looks up your domain’s MX records
2. MX records point to mx.momail.io
3. The Email Worker receives the incoming message via SMTP

2. Initial Processing

The Email Worker performs several tasks:

• Parsing: Extracts headers, body, and attachments
• Validation: Verifies the sender and checks for spam
• Threading: Groups related emails into conversation threads
• Metadata Extraction: Extracts sender, recipients, subject, dates

3. Database Storage

Email data is stored across multiple tables:

Table	Purpose
emails	Core email content and metadata
email_threads	Conversation thread information
email_attachments	Attachment metadata and storage references
mailboxes	Mailbox configuration and status

4. Vector Embedding

For AI-powered search, MoMail creates vector embeddings:

1. The email body text is extracted and cleaned
2. Text is split into chunks (typically 512 tokens)
3. Each chunk is converted to a vector embedding
4. Vectors are stored in Cloudflare Vectorize for similarity search

This enables semantic search queries like:

• “Emails about the Q4 budget from Sarah”
• “Find messages regarding the project delay”
• “Show me invoices from last month”

5. Webhook Notifications

If you have configured webhooks, MoMail will:

1. Build a webhook payload with email metadata
2. Sign the payload with your webhook secret
3. POST to your configured URL
4. Retry on failure (up to 3 attempts with exponential backoff)

Data Retention

MoMail maintains different retention policies based on your plan:

Plan	Email Retention	Vector Retention
Free	30 days	30 days
Pro	1 year	1 year
Enterprise	Unlimited	Unlimited

After the retention period:

• Email content is permanently deleted
• Vector embeddings are removed from search
• Thread metadata is retained for reference

Privacy and Security

• All data is encrypted at rest using AES-256
• TLS 1.3 is required for all connections
• Access is scoped to your account via API keys
• Regular security audits and penetration testing

Monitoring the Pipeline

You can monitor your email pipeline through:

1. Dashboard: Real-time statistics and recent emails
2. Webhooks: Receive events for pipeline stages
3. API: Query email status and delivery information

# Check recent emails
curl https://api.momail.io/v1/search \
  -H "X-API-Key: your_key" \
  -X POST \
  -d '{"query": "*", "limit": 10}'