Deploying Hundreds of Thousands of SEO Pages with a Few Lines of Code in Rails

Deploying Hundreds of Thousands of SEO Pages with a Few Lines of Code in Rails

When I joined Hired about 3 years ago, we basically had a Home page, About page, and not much else.  Recently we’ve decided to dig deeper into SEO and renew our focus on driving organic traffic.

This whole time, I’ve wanted to work on SEO to leverage all of the content we have in a way that would allow us to reach candidates and employers searching for jobs or candidates matching certain criteria. We have tens of thousands of companies on our platform and an order of magnitude more jobs in our database, so we needed to come up with a way to leverage that data. 

Looking at the way people search, we decided to start down a path of figuring out how to programmatically build new pages to target searches like “ruby on rails jobs san francisco” or “biotech companies in austin”.   We could have gone with a search-like product where filters could be applied as parameters like /jobs?city_id=1&industry_id=2, but those urls would be worthless for SEO purposes.  We wanted to generate and open up landing pages with urls like /jobs/san-francisco/biotech.

The MVP

When we first began building these pages as an MVP, we started with the most simple version possible: a few lines in the routes file, some constraints, a few new actions, and a bunch of duplicated code.  Here’s a basic example:

First we’d add a route:

Then in our controller

end
Pretty straightforward right?

Adding More Pages Unsustainably

Once we had a basic idea of how these might perform, we decided to go further by adding more top level filters, like state, region, category, industry, required skills, role, and more.  So to do that, we started off going the easy route: copy+paste! 

All of a sudden our routes file started to look like this:

This started to look worse when we decided to also filter companies as well, duplicating each line for companies and jobs… The controllers were looking even worse than the routes as we’d create a new action for each filter and write lookup code for each filter combination 

AND we needed to create a new constraint for every single combination… There was duplication everywhere!  Yikes!

It Gets Worse

Things were already looking messy enough, but then we decided we wanted to build pages with nested filtering like [Find Backend Engineer Jobs at Gaming Companies in San Francisco](https://hired.com/jobs/san-francisco-ca/gaming/backend-engineer)… Did we really need to manually add routes and a controller action for every combination?  What about nested filters?  Something like

started to provide more and more problems.  This was clearly no good.

Rethinking What We’re Doing

So I decided to re-evaluate what our SEO pages were actually doing:  basically all we wanted to do is take a resource like Job, apply some set of arbitrary filters matching a url pattern like city/industry, and display those with some custom copy here and there for the filter pair, exposing the urls in our sitemap. 

So I started by defining a list of url patterns that effectively would also define the filters:

Now in our controller method, we just need to forward the request parameters which contain our filters:

Voila!

With this approach, we can now create tons of pages by adding a new pattern to the /lib/seo_routes.rb file.  If you think about a pattern like /:market/:industry/:role and consider we have 16 markets, 43 industries, and 44 roles, just adding this single pattern generates 30,272 new pages!!

Some Notes

Obviously the code shown above is simplified.  In the controller, we also do some stuff like setting instance variable names, and interpolating the variable names into I18n calls to set the meta descriptions and titles and headers, etc.  We can also use the PATTERNS list to iterate across all the matching records to build our sitemap pretty easily.

candidate-banner-2