Advanced Log File Analysis – Interview with Bastian Grimm

Our Search Elite event is for those in technical SEO and those who want to learn more about it. We have lots of great speakers and I am pleased to be able to interview them before the day itself. I spoke to Bastian Grimm, Director of Organic Search at Peak Ace to find out the background and the work that goes into log file analysis.

1) You are going to talk about Advanced log file analysis. Can you share how a person who is new to log files analysis can first get into this line of work? For example, do they need to come from a certain background or use a certain tool?

Sure! So generally, there are loads of different types of log files out there. For SEO purposes, what we really need, are so-called “access logs” – which are created and populated by the web server in-use (e.g. Apache, nginx, IIS, etc.). Those log files are essentially documenting every request to the server and can contain a variety of different information (depending on configuration). For example, HTTP status codes, user-agents, the requested URL, and so on and so forth. This is all fairly technical, so from a background perspective, it certainly helps to understand the basics of how the internet is built. This includes a deeper understanding of what a web server is, how the HTTP protocol works, etc.

So yes, any sort of IT-related background makes it easier, but certainly, it’s also possible without it. It just might take a bit more reading on specific topics. I’ll be covering the tool side of things as well: since log files essentially are only basic text files and, depending on a websites size and the number of requests, can become pretty big quite quickly. Working on log files without a proper toolset just won’t get you anywhere.

2) How did you get into log file analysis and when?

For the technical SEO team at Peak Ace, log file analysis is an essential part of our technical SEO consulting services. We almost always combine it when working on any type of technical site audit. Just using crawl data is simply not enough anymore – since all tools out there are essentially only “simulating” what search engine crawlers are doing; but only access logs do really reflect how Googlebot (and other crawlers) handle a website, what URL they are accessing and how often.

3) What three issues of a site can you get from log file analysis that you cannot get in any other way?

There is really loads of stuff that you can get out of a properly done log file analysis. Some of the items I’ll be covering at Search Elite will be how to:

  1. Spot crawl hubs and utilise those for better internal linking / more efficient discovery of fresh contents.
  2. Identify crawl waste, e.g. caused by weird parameter configuration.
  3. Identify errors – during a migration – straight away (since Google Search Console data comes in with a huge delay).

One of the most important things when looking at log file data is to have the right questions in mind before you actually get started. It’s easy to get lost in the vast amount of data, so an idea what to look for or what topic to tackle (e.g. “let’s only look for crawl errors and what causes them” during one session) can make things way easier and more straightforward.

4) How long would you recommend a consultant at an agency or in-house SEO (client side) spends on log file analysis per week?

There is more than one answer to the question. In my opinion, the most significant variables are most definitely the size and complexity of a website, the tool setup, availability of log files as well as the individual task at hand. For example, from a monitoring perspective and once you have your dashboards set up properly, this can just be a couple of minutes to ensure there is nothing out of the ordinary. Bigger tasks I’d approach as dedicated projects, with a reasonable scope and concept in mind.

Thank you for your time Bastian, we look forward to seeing you at Search Elite and learning more about log file analysis.

127 thoughts on “Advanced Log File Analysis – Interview with Bastian Grimm

Leave Comment