Drupal SEO Pitfalls and How To Avoid Them

Drupal SEO Pitfalls
and
How To Avoid Them
- by Brent Gees and Wouter De Bruycker

Drupal SEO Pitfalls and How To Avoid Them - DrupalCon Amsterdam 2019 3
ABOUT US
Brent Gees
● Drupal Architect and Drupal Trainer at
Dropsolid
● Speaker at Drupal Dev Days Lisbon 2018
● Speaker at Drupalcamp Ghent 2018

ABOUT US
Wouter De Bruycker
● SEO Strategist & Evangelist at Dropsolid
● Speaker at Digital Content Marketing
Congress: "Technical SEO 101 (& 102)"
● Closing Keynote Speaker at DMB Day
“Today is Yesterday's Future: SEO in
Evolution"

2PUBLIC ENTITIES
ALL PAGES SHOULD BE AN ENTITY
ABOUT US
3
1

PUBLIC ENTITIES
By default, entities might be publicly available on their own unique
URL.
Example: a “team member” module could generate a public node for
each team member, while they are really only used for a “team overview
page” and the site doesn’t need detail pages for each team member.
https://mysite.com/node/42, https://mysite.com/taxonomy/term/42, ...
The case

PUBLIC ENTITIES
This results in low value and “thin content” pages, indexable by Google.
You don’t want these pages on your site because they are a waste of
resources (bandwidth, crawl budget, database storage, …).
The problem

PUBLIC ENTITIES
Prevent those entities from being accessed by visitors
This can be done by modules such as rabbit_hole
The solution

3ALL PAGES SHOULD BE AN ENTITY
INDEXED INTERNAL SEARCH
PUBLIC ENTITIES
4
2

Oftentimes, some pages on websites are generated based on other
content. These pages are not an “editable node” in the backend.
Examples: homepage, overview pages, ...
The case

When a page isn’t editable as a node, there’s no easy way to edit the
meta tags, or to conﬁgure the XML sitemap inclusion for this page, or ...
The problem

• Use the core drupal layout builder to construct the homepage
The solution

• Use paragraphs together with modules such as block_ﬁeld or
overview_ﬁeld to add blocks to the page.
The solution

4INDEXABLE INTERNAL SEARCH
INDEXED TEST ENVIRONMENTS AND PAGES
5
3

INDEXABLE INTERNAL SEARCH
By default, internal search result pages are often indexable by search
engines.
The case

This results in low value and “thin content” pages, indexable by Google.
You don’t want low-value pages in the index.
Google also implicitly mentions in its quality guidelines to not let
Googlebot index internal search results.
The problem

When using views for your search pages
install metatag and metatag_views module
The solution

When using a node for your search pages
Install metatag and add the ﬁeld to your content type
The solution

5INDEXED TEST ENVIRONMENTS
AND PAGES
ASSETS BLOCKED BY ROBOTS.TXT
6
4

Development and staging environments are often crawlable and
indexable by search engines because of conﬁguration issues.
Or because of laziness ¯_(ツ)_/¯
Temp content is also something that often shows up in search results.
The case

Everybody probably agrees we don’t want our staging environments or
paragraph testing pages to be in the Google index, or even be publicly
available, for a wide range of reasons.
The problem

INDEXED TEST ENVIRONMENTS AND PAGES The problem

Test pages on live environments
Unpublish if possible or prevent from indexing with metatag module
The solution

Test environments
Use htpassword protection for your environments
The solution

6ASSETS BLOCKED BY ROBOTS.TXT
MODULE OVERLOAD
7
5

Sometimes website assets (like favicons or images) are located inside a
folder that’s blocked for crawlers via the robots.txt.
The case

We want Google to understand our
entire page and “see” it as a regular
website visitor would see it.
If some assets are blocked via
robots.txt, Google can’t view the
page as if it was a regular website
visitor.
The problem
Image: notiﬁcation in Google Search
Console due to blocked site assets.

Make sure your website assets (images, icons, favicon, …) are in a
publicly available folder, not blocked by robots.txt.
Keep an eye on your Google Search Console property for notiﬁcations
regarding blocked resources.
The solution

7MODULE OVERLOAD
REDIRECT-REDIRECT-REDIRECT
8
6

MODULE OVERLOAD
There are a lot of modules available
for a wide range of use cases,
which is mostly a good thing.
The case

MODULE OVERLOAD
Be careful not to overload your website with bulky or unnecessary
modules.
Each one could require multiple resources to be downloaded and
executed by the browser, which will have an impact on your website
speed.
The problem

MODULE OVERLOAD
“We encourage you to start looking at your site's speed “...” — not only to
improve your ranking in search engines, but also to improve everyone's
experience on the Internet.”
Source: Google Webmaster Central Blog - “Using site speed in web search ranking”
https://webmasters.googleblog.com/2010/04/using-site-speed-in-web-search-ranking.html
The problem

MODULE OVERLOAD
Think twice before installing a module. Do you really need it or is there an
easier solution?
Check if there are unused modules on your site with something like
unused_modules.
The solution

8REDIRECT-REDIRECT-REDIRECT
SECURITY LEAKS IMPACTING SEO
MODULE OVERLOAD
9
7

Not paying close attention to redirect setups could result in multiple
redirects following each-other.
The case

Redirect chains are not search-engine-friendly.
When Googlebot visits a page which returns a redirect statuscode, it adds
that page to the bottom of its “to visit” list for that website.
For small websites this is not a big problem, but for big websites this
could result in slow indexation times.
The problem

You want as little redirects as possible.
The problem

Don’t just uncomment the lines in htpasswd
The solution

But think along with your site
The solution

9SECURITY LEAKS IMPACTING SEO
ROBOTS.TXT DISALLOW != NOINDEX
10
8

Allowing public ﬁle-uploads could result in lower organic trafﬁc when not
setup correctly.
The case

If ﬁles can be uploaded without some form of authentication or
CAPTCHA, this could result in thousands of ﬁles being uploaded by
spammers, and indexed by Google.
The problem

SECURITY LEAKS IMPACTING SEO The problem

When Google notices spam on your
website, they could punish you with
a “manual action”, potentially
lowering your organic search
visibility.
The problem

“Google issues a manual action against a site when a human reviewer at
Google has determined that pages on the site are not compliant with
Google's webmaster quality guidelines”
“If a site has a manual action, some or all of that site will not be shown
in Google search results.”
Source: https://support.google.com/webmasters/answer/9044175?hl=en
The problem

Use a module to implement reCAPTCHA, recaptcha or simple_recaptcha
The solution

Know when to use private ﬁles and place them outside of Drupal’s web
root.
The solution

10ROBOTS.TXT DISALLOW != NOINDEX
GOOGLE ANALYTICS HORROR
11
9

Contrary to popular belief, blocking a page (or folder) via robots.txt or
adding a noindex directive to the meta robots tag are not the same thing.
Robots.txt instructions impact crawling, not indexing.
Noindex directives using a meta tag impact indexing, not crawling.

It sounds weird, but Google is able to stumble upon a link on an external
website linking to a page blocked by robots.txt, and still index it.
The result will most likely be a snippet in the search results without a title
or description, since Google can’t read the title or meta-description.

“Now wait a second Wouter. I understand it could happen in theory a link
is indexed even though it’s blocked by robots.txt, but do things like this
really happen in practice?”

Why yes, yes they do:

A weird “/home//home” page is indexed on drupal.org.
Let’s take a look at the robots.txt: https://www.drupal.org/robots.txt …
Bingo:
They noticed “strange homepage URLs” and tried removing
this page from the Google index by blocking Google from
crawling the page.

11GOOGLE ANALYTICS HORROR
RAPID-FIRE BEST PRACTICES
12
10

Correct data is very important when analysing SEO-efforts
Pay close attention to sudden drops and spikes in Google Analytics data.
There might be a conﬁguration issue.

GOOGLE ANALYTICS HORROR The case

https://www.drupal.org/project/eu_cookie_compliance updated
Each pageview started a new session as long as website visitors didn’t
accept the cookies
The problem

Two-step ﬁx:
The solution
1. Anonymize the visitor
IP-addresses (GA or GTM setup)
2. Whitelist the Google Analytics
cookies (Drupal module setup)

GOOGLE ANALYTICS HORROR The solution
Update/patch the google analytics module to comply with the
eu_cookie_compliance so that the snippet isn't loaded until approval was
given.
More information:
https://www.drupal.org/project/google_analytics/issues/3060312

12RAPID-FIRE BEST PRACTICES
GOOGLE ANALYTICS HORROR 11

Use the Google Analytics module (google_analytics) or the Google Tag
Manager module (google_tag).
Not both.

Aggregate and minify css and js ﬁles where possible.
https://www.drupal.org/project/advagg

Make sure each page has a correct/well-configured canonical tag.
Not enough time to go into detail because of the complexity (what about paging parameters?
facet filtering parameters? combinations with hreflang? ...)
come find us after the talk or search the interwebs for resources :)

Use Pathauto module to generate nice urls for your pages

Follow-up on the amount of pages indexed by Google via Google Search
Console.
Seems too high?
Maybe some rabbithole setup is
needed to remove separate pages
from the index.
Seems too low?
Maybe important pages are
no-indexed?

Make sure your base url is not ‘http://default/’ in your sitemap XML,
we’ve seen this issue occur a lot in live sites.
Setup your cron job correctly, more info:
https://www.drupal.org/project/xmlsitemap/issues/1944518

Create checklists!
• Things that have to be done before going live
• Things to check when site is live
• Things to check when site is live for x days

Image: WARNER BROS. ENTERTAINMENT INC.

QUESTIONS
Time left?
Fire away!
No more time?
Come ﬁnd us at the Dropsolid
booth (#13)
Get in touch:
https://www.linkedin.com/in/brentgees/
https://www.linkedin.com/in/wouterdebruycker/
+ Keep an eye on the Dropsolid Twitter feed for these slides!

Drupal SEO Pitfalls and How To Avoid Them - DrupalCon Amsterdam 2019
TODO
ABOUT US
70
Join us for contribution opportunities
Thursday, October 31, 2019
Mentored
Contribution
9:00-18:00
Room: Europe Foyer 2
First Time
Contributor Workshop
9:00-14:00
Room: Diamond
Lounge
General
Contribution
9:00-18:00
Room: Europe Foyer 2
#DrupalContributions

Drupal SEO Pitfalls and How To Avoid Them - DrupalCon Amsterdam 2019
TODO
ABOUT US
71
What did you think?
Locate this session at the DrupalCon Seattle website:
https://drupal.kuoni-congress.info/2019/program/
Take the Survey!
https://www.surveymonkey.com/r/DrupalConAmsterdam
Thank You!

Drupal SEO Pitfalls and How To Avoid Them

Recommended

Recommended

More Related Content

Similar to Drupal SEO Pitfalls and How To Avoid Them

Similar to Drupal SEO Pitfalls and How To Avoid Them (20)

Recently uploaded

Recently uploaded (20)

Drupal SEO Pitfalls and How To Avoid Them