Do you know that?
Your rankings climb incessantly, you pat yourself on the back.
And then…
The damn rankings suddenly DROP.
You look at the page and after an eternal search you realize that some settings from the staging environment were transferred during a release or that someone played on the metas.
Pretty annoying!
And just to make sure that doesn’t happen, top SEOs have some important things monitored on a daily basis. So they are always in the picture and can react before the rankings fall.
SEO Check 1: robots.txt
The horror scenario of every SEO: You build a really cool new site, check everything in the staging environment and look forward to faster loading times, more conversions and higher rankings.
And then nobody checks the robots .txt after the release.
Checking means:
- Bots have access to the website
- Bots do not have access to specific directories
User-agent:*
disallow:/category1
allow:/category1/product1
This special directory protection can be monitored with very few on-page crawlers.
In URL-Monitor you simply enter the text of your robots.txt and get an info if there is a change.
Or at least you will receive information if the robots.txt is no longer available or has been moved to another directory via 301:
After all, 10 of the 100 most visited websites have done so in the last 30 days:
A little tip to the agency people: This check alone can bring you umpteen new customers ;)
SEO Check 2: Domain Redirects
Refer http to https pages?
Refers to www. on non-www?
Are pages redirected with a trailing slash?
Are case-sensitive pages redirected?
A small, unwanted change in the htaccess and the work is done: All URLs can be found twice on the net.
Simple remedy: Store URLs in URL Monitor and let them be informed if they no longer redirect.
SEO Check 3: Meta Robots Tag
As with robots.txt, it must be ensured at all times that bots are allowed to crawl and index the page.
Since staging environments are often set to noindex – instead of protecting them from access via htaccess – you have to make sure that the most important pages are indexable at all times.
So have them monitored daily and play it safe.
SEO Check 4: Canonical Tags
If you have pages that intentionally refer to others via canonical, have them monitored.
With all other sites, you simply monitor whether canonicals are set and constantly present.
In fahrrad.de, for example, the canonicals regularly disappear to be present again later:
SEO Check 5: Meta Titles & Descriptions
You are the SEO. You specify what Meta Title and Description should look like.
But suddenly there is a new agency or the staff in a department has been replaced.
They are used to making changes to the website themselves.
Thus, a title of the CTRs of 30% and more becomes a one-word title that no one is interested in.
Sounds impossible to some and yet it happens constantly and with projects of any size.
It is also essential that you immediately see major changes. Here, for example, that fahrrad.de on 15.06. has changed all descriptions:
SEO Check 7: Constant Status Codes
301 redirects and with 410 deleted pages are cached by Google and some browsers.
URLs with this status code will sooner or later fly out of the index and will never be crawled again.
If you want to detect these types of errors, you should monitor special URLs daily. Just as we did in the case of emp.de (see screenshot).
A few months ago, we had imported all the top rankings of the shop into URL-Monitor. Now we see that a URL with a good ranking should be completely deleted from the index by a 410 status code.
After some time, it should then be indexed again and deleted again.
As you can see, this change has done lasting damage to the rankings:
With ikea.com, pages are occasionally redirected by 301 in order to go back online a little later.
In this case, https://www.ikea.com/us/en/cat/storage-organization-st001/ was made on 01.06. redirected by 301.
Whether they use URL monitor to see this change :thinking-face:
SEO Check 8: Search Pages
If your search pages rank, such as obi.de, it makes sense to check the search URLs regularly.
If there are changes to the search for an update or if products are removed from the range, a 404 error occurs and potential customers go nowhere. For example, with https://www.obi.de/search/plexiglas/.
SEO Check 8: Rankings
Rankings only check for 8th place?
Absolutely!
Nothing is worse than having to react to fallen rankings. You carry out all previous checks to ensure that technically everything runs smoothly.
This is all the more true since ranking checkers always have a time delay:
- SearchConsole: 1 – 2 days
- Sistrix, Xovi: min 1 day
- metrics.tools: up to 7 days
- ahrefs: min 1 day
You only react when the most important metric is already declining and customers and colleagues are sitting in your neck.
Result
You should always keep an eye on certain things so that you can react immediately.
And with URL Monitor, it’s a breeze to have your eyes everywhere.
Share This Story, Choose Your Platform!