MongoDB to Convex: A Step-by-Step Data Migration Script

Github Repo

merge-mongodb-convex

Examples of stream merging data from MongoDB to Convex

In these examples, the MongoDB schema was different from the schema created in Convex. This repository provides scripts to help migrate your data while handling schema transformations.

Useful Resources

For detailed explanations on MongoDB to Convex migration strategies, check out these articles:

• Convex Documentation
• Convex Search
• MongoDB to Convex: A Step-by-Step Data Migration Script
• How I Wrote a Script to Migrate Data from MongoDB to Convex (Dev.to)
• How I Wrote a Script to Migrate Data from MongoDB to Convex (Medium)

Migration Scripts

The repository contains several scripts for different aspects of data migration:

Setup

1. Install dependencies:

npm install mongodb convex dotenv

2. Configure your environment variables:

MONGODB_URI=your_mongodb_connection_string
CONVEX_URL=your_convex_deployment_url
CONVEX_AUTH_TOKEN=your_auth_token

Available Scripts

1. Users Migration (users.js)

Migrates user accounts from MongoDB to Convex:

node users.js

• Handles email normalization
• Checks for existing users
• Updates or creates users as needed

2. Organizations Migration (organizations.js)

Migrates team/organization data:

node organizations.js

• Transfers team structure
• Handles member roles and permissions
• Generates join codes and slugs

3. Contacts Migration (contacts.js)

Migrates contact information:

node contacts.js

• Handles phone numbers and email formatting
• Processes related contacts
• Manages contact tags
• Links contacts to tasks

4. Properties Migration (properties.js)

Migrates property records:

node properties.js

• Handles address formatting
• Processes property metadata
• Links properties to contacts
• Manages property tags and tasks

5. Tag Management

Two scripts for handling tags:

node search_contact_tags.js  # Find and remove duplicate tags
node update_tags.js         # Update tag structures and relationships

6. Record Updates (update_records.js)

Updates existing records with new data:

node update_records.js

• Updates activities and notes
• Links records together
• Maintains timestamps

Migration Process

1. Preparation

   • Back up your MongoDB database
   • Set up your Convex project
   • Configure environment variables

2. Order of Migration

   # 1. First migrate base data
   node users.js
   node organizations.js
    
   # 2. Then migrate main records
   node contacts.js
   node properties.js
    
   # 3. Finally, handle relationships and cleanup
   node update_tags.js
   node search_contact_tags.js
   node update_records.js

3. Verification

   • Check migrated data in Convex dashboard
   • Verify relationships between records
   • Test application functionality with migrated data

Error Handling

All scripts include:

• Retry mechanisms for failed operations
• Batch processing to handle large datasets
• Error logging and progress tracking
• Ability to resume from last successful operation

About Data Migration

When migrating from MongoDB to Convex, there are several key considerations:

1. Schema differences between MongoDB and Convex need to be handled
2. Data transformation may be required during migration
3. Convex provides built-in migration tools and patterns for handling large datasets

For more information about Convex migrations, you can also check out:

• Migrating Data with Mutations
• Lightweight Zero-Downtime Migrations

Contributing

Feel free to submit issues and enhancement requests!