SEO - Search Engine Optimization

Tackling Tag Sprawl: Crawl Budget, Duplicate Content, and User-Generated Content

Last Updated: May 24, 2017

Posted by rjonesx.Alright, so here’s the situation. You have a million-product website. Your competitors have a lot of the same products. You need unique content. What do you do? The same thing everyone does — you turn to user-generated content. Problem solved, right? User-generated content (UGC) can be an incredibly valuable source of content and organization, helping you build natural language descriptions and human-driven organization of site content. One common feature used by sites to take advantage of user-created content are tags, found everywhere from e-commerce sites to blogs. Webmasters can leverage tags to power site search, create taxonomies and categories of products for browsing, and to provide rich descriptions of site content. This is a logical and practical approach, but can cause intractable SEO problems if left unchecked. For mega-sites, manually moderating millions of user-submitted tags can be cumbersome (if not wholly impossible). Leaving tags unchecked, though, can create massive problems with thin content, duplicate content, and general content sprawl. In our case study below, three technical SEOs from different companies joined forces to solve a massive tag sprawl problem. The project was led by Jacob Bohall, VP of Marketing at Hive Digital, while computational statistics services were provided by J.R. Oakes of Adapt Partners and Russ Jones of Moz. Let’s dive in. What is tag sprawl?We define tag sprawl as the unchecked growth of unique, user-contributed tags resulting in a large amount of near-duplicate pages and unnecessary crawl space. Tag sprawl generates URLs likely to be classified as doorway pages, pages appearing to exist only for the purpose of building an index across an exhaustive array of keywords. You’ve probably seen this in its most basic form in the tagging of posts across blogs, which is why most SEOs recommend a blanket “noindex, follow” across tag pages in WordPress sites. This simple approach can be an effective solution for small blog sites, but is not often the solution for major e-commerce sites that rely more heavily on tags for categorizing products. The three following tag clouds represent a list of user-generated terms associated with different stock photos. Note: User behavior is generally to place as many tags as possible in an attempt to ensure maximum exposure for their products. USS Yorktown, Yorktown, cv, cvs-10, bonhomme richard, revolutionary war-ships, war-ships, naval ship, military ship, attack carriers, patriots point, landmarks, historic boats, essex class aircraft carrier, water, ocean ship, ships, Yorktown, war boats, Patriot pointe, old war ship, historic landmarks, aircraft carrier, war ship, naval ship, navy ship, see, ocean Yorktown ship, Warships and aircraft carriers, historic military vessels, the USS Yorktown aircraft carrier As you can see, each user has generated valuable information for the photos, which we would…

Source: Tackling Tag Sprawl: Crawl Budget, Duplicate Content, and User-Generated Content

About the author / 

S K Routray

S K Routray is a computer science graduate and Co founder at Gracioustech.com. He worked as a Online Marketing lead at many MNC Companies. He has passion for writing on SEO techniques, Social Media Marketing and digital marketing techniques. If he wasn’t an online marketer, he'd take his love for food and become a great chef cum hotel entrepreneur. Join NAS Writers team to write for NAS.

Email Subscriptions

Enter your email address:

Delivered by FeedBurner

Subscribe to our Newsletter

Best Email Marketing Tool!

Multiply Profits AND Automate Your Business

AWeber's email marketing software makes it easy.

Learn how they can do it for you, too.

Follow us on Twitter