How We Forked AngelList

In this blog post, I’ll describe how we divided a massive codebase, a high-traffic domain, and infrastructure between AngelList and Wellfound (formerly AngelList Talent).

We split the process into multiple stages to represent each of the separate layers:

• Hosting infrastructure
• Codebase
• Domain
• Data stores

Infrastructure Separation

First, we wanted to split the hosting & CD/CI infrastructure into a new AWS account.

Most of our services are deployed to a Kubernetes cluster using AWS EKS. This includes web servers, on-demand background processing workers, cron jobs executed by schedule, etc.

We “cloned” the existing Kubernetes cluster and created a new CI/CD pipeline by using Buildkite and ArgoCD. After carefully configuring permissions, ingress rules, and our service mesh, we had two identical production environments running on different AWS accounts.

This deployment setup allowed us to start testing things early by splitting traffic between these two AWS accounts. We used NGINX Ingress and configured routing rules based on URL paths to determine which AWS account to hit. Finally, we finished this setup by migrating our background workers that listen to the (still) shared Redis cluster and cron jobs orchestrated with Argo Workflows.

The fun had begun!

Codebase Forking

AngelList has many applications and services. Our biggest and oldest repository was shared with Wellfound and housed multiple “applications” open to the internet (mainly a large Rails application with a few Next.js applications on the frontend).

Forking a repository can be accomplished with just a few clicks on GitHub, but one of the main challenges is dealing with any remaining shared-through-the-code resources — specifically routes and database migrations.

For shared routes, we added new URLs by splitting the existing functionality or duplicating some of the URLs to have a clear separation between AngelList and Wellfound paths.

For database migrations, we considered multiple different options, such as creating two different main branches, creating a bigger monorepo with two monorepos inside, using tools like bit, extracting shared code into a library, running migrations by using a separate service, or using git submodules and trees.

Our goals were to try to decouple as much as possible, have two separate repositories within two separate GitHub organizations, and avoid making development workflows more complicated. After quickly trying and testing different approaches, we decided to simply sync the migration folder between two repos by using a GitHub Action (for example, Repo File Sync Action). When a new migration is added to one of the repos, it’ll be automatically committed to the other.

Forking the codebase was a huge productivity boost for both companies. This allowed us to start doing what engineers like the most… removing lots of unnecessary code and libraries!

As a result, we were able to speed up our CI/CD builds by more than 2x, iterate and release changes faster, improve code maintainability, implement lots of company-specific optimizations, and reduce infrastructure costs.

Domain Migration

At AngelList, engineers move with urgency by scoping down. We didn’t want to spend multiple months preparing and hoping to perform one giant migration at once. Instead, we wanted to move faster and use an iterative approach. To do that, we implemented a header-based feature flag that allowed us to test both logged-in and not-logged-in user flows. We asked our internal team members to install a browser extension (for example, ModHeader) and send a custom header like "X-Please-Surprise-Me: true" to enable the new domain.

After that, we had to update different parts of our system that depend on the domain:

• Our codebase with deeply hardcoded "angel.co" constants
• Log in with social accounts through OAuth workflows
• Third-party browser services such as Google Analytics and Segment
• Embedded with domain restrictions content such as Vimeo
• SEO metatags and sitemaps to avoid duplicate pages
• Our Terms of Service and Privacy Policy
• .. (and more)

One of the most interesting challenges was redirecting users to the new domain while maintaining their active login sessions. There are many ways to accomplish this, but choosing a solution depends on the existing authentication workflow. Here is what ours looked like at a high level:

Because we still shared the data store with sessions, our main goal was to set the session cookie in users’ browsers for the new domain. We did it by implementing redirects with temporary tokens:

1. Generate a one-time token for a session ID and pass it to the new domain
2. On the new domain, fetch the session ID and set it in a cookie
3. Visit the requested path on the new domain and authenticate by using the session cookie as usual

These redirects happen seamlessly in users’ browsers. There is no need for additional inter-service message communication and encryption/decryption. Once the session cookie is set on the new domain, the regular authentication workflow works as usual.

For public pages visited by not logged-in users and search engine crawlers, this looks much simpler:

We successfully migrated to the new domain on November 8, 2022. This unlocked a path towards clearer brand separation and lower customer confusion.

What’s Next

We’re currently migrating emails we send to customers from @angel.co to @angellist.com. This is a gradual process as it requires warming up our new dedicated email IP addresses.

Afterwards, our plan is to complete the last stage: separating data stores. We’ve already separated our secondary data stores (e.g. Elasticsearch), but we still need to migrate our main MySQL databases. This is a complex and irreversible process that we’ve started planning, but haven’t yet executed.

If you want to help out and work on our mission to accelerate innovation, we’re hiring!