Changed URL structure (via 301 redirects) – seen a 25% traffic drop, how long to recover?
Long time lurker and first time poster – i'm hoping some of you maybe able to help.
We have built our website over the past 15 years or so and have a very good domain authority and rank well for a lot of terms. We decided to update our URL structure to make it more descriptive and improve our internal linking (moving from www.sitename/subfolder/keywordterm to www.sitename/category/keywordcategory/keywordterm). We employed a well respected independent SEO Consultant and have been working with them for the past couple of months to put this in.
We have followed all of the best practices around this and have performed significant testing and have validated that the 301 redirects (for our 5000 or so URL's) are all working correctly (with single redirects for all URL's). Since this release just over a week ago we have dropped around 25% of our traffic and this appears to be growing (in terms of losses) by the day. We are seeing our keywords update to the new URL's however drop from position 1, 2 or 3 right the way down to position 90 and then back up to 1,2 or 3 again as Google looks to be trying to figure out what we have done (Google Dance effect maybe?).
What I am trying to understand (I now realise it was probably the wrong thing to update our URL's) is how long Google will take to understand the new structure of our site and at what point we should expect the rankings to go back to normal (if indeed they will return to normal). I have read a number of posts online and within Reddit which indicates that this could be anytime from a month to 6 months.
Has anyone gone through anything similar and can advise on how long we will likely continue to experience flux in the rankings? Note: Google appears to have crawled our site in it's entirety as we can see significant uptick in the number of crawls being performed on the site which now seems to have slowed to normal levels.
Thanks for any help any of you can offer.
It can take a while for something like this to completely resolve, but it my experience it should look more like a quick drop and a gradual recovery as opposed to a gradual loss. That said, one week is a short period of time, so that doesn't necessarily indicate you have a more serious problem.
If you've verified your 301s are all working properly, you might also want to do a scan with Screaming Frog or a similar tool and update your internal links (especially in your main nav) from the old URLs to the new, even if they forward correctly currently.
Ok thanks – do you have any experience in terms of this happening with sites that you have worked on / experience of sites this has happened to? We are trying to build up a picture of timeframes for recovery. Thanks.
Not this scenario specifically, but I've been a part of site migrations that involve URL updates. As far as I know, there's no rule about how long it takes to get back to "net zero," but as I said, my experience is that any dropoff happens quickly and early on, and then should turn to a positive trend if no technical problems exist.
Often, this can be caused by some unintended side effect: loss of robots.txt directives, some onpage detail changed when you edited the URLs, change in the link architecture on the site. I have seen organisations spin up a legacy version of the site from backups/version control to run comparative crawls.
Assuming that's not the case, and you really did just change URLs. I wouldn't be so quick to chalk it up to 'the Google dance' – migrations are about the riskiest thing you can do as an SEO, and many go wrong.
A couple of investigation ideas off the top of my head
First, verify that you have actually redirected all the URLs that Google is trying to crawl. Pull the logs, look for any recent errors, check that they all redirected to the correct destination.
Crawl all of the old urls and really hammer them (crawl with 50 threads/hundreds of urls/sec) to ensure that your redirection is consistently working and there's no random servers running legacy code/performance issues that cause intermittent errors.
Second, Google is known to assign value to directories (or more correctly, Google is known to identify and quarantine bad directories). Shift a handful of pages back to their old url and see what that does for you.
Double check that all of the backlinked destination URLs in that directory actually redirect to the correct destination. Verify that no one's done any crazy disallowing as part of this redirecting.
Edit: nevermind, just read the "and improve our internal linking" note. That is the scary type of thing that makes so many migrations fail. I would put all of my effort into going down that rabbit hole.
Hopefully your respected SEO consultant doesn't also find time to speak at a hundred conferences a year.
Thanks for taking time out, you raise some good points here.
Robots.txt looks fine, we have run a lot of ScreamingFrog scan's against the old codeset vs the new codeset and all that looks good to us, albeit we will perform some further checks here.
We will try the hammer'ing approach you describe as that raises an interesting point around performance. I also like the idea of maybe moving a handful of url's back to the old structure to see how they perform, albeit I'm a little concerned that might confuse Google further.
I'll start checking some of the backlinks to ensure that they are linking correctly and working as expected.
One thing to note as part of this work, we also optimised the pages slightly so that they have in some cases subtly revised H1's and also the internal linking structure for these pages has changed in that they are now linked with less depth (previously 4 now 3 and sometimes less if we link certain pages direct from the home page). We also added structural data to these pages in the form of breadcrumbs (using JSON-LD). So you're right it wasn't just a straight url migration albeit that seems to us to be the most obvious element to have caused the issues.
Internal link equity is a tricky thing – it's less about the level that the links first appear on, and more about the graph of links across your entire site (and the web to a lesser extent). I have seen subtle changes in links cause massive shifts in ranking, and I've also seen big changes have almost no effect.
It could be some unintended side effect of your optimizations, it could be something completely unrelated, or nothing at all.
Thanks for your notes here, we have started working through a number of your suggestions.
Your new URL structure is pretty spammy, and there's no guarantee that you'll come back to where you were.
I'm all for what this guy says. If you're making a site-wide change to get better Search Engine Optimization (SEO), why would you even think that Google is going to reward you? Why would think that Google can't see what you're doing?
I have never made a site-wide change to my URL structures on any of my websites because I tend to follow the mantra that if Google thinks you're trying to game their algorithm, they're going to game you back. Just stick with what you got, and focus on content, headings, subheadings. Moreover, keep your website on topic. I think most of why people are seeing drops is because they are watering down their website by trying to expand horizontally. Meanwhile, Google seems to be rewarding websites that grow more narrow and more vertically. Just food for thought.
It is quite common to change URLs while restructuring a website and in my experience. And max it takes 3 to 4 weeks to get back to actual traffic (mostly it is a week to 10 days).
Also, I never change all URLs in one go to avoid giving shock to bots that something drastic has changed on the website. Ideally, go category-wise over a period of time apart from other strategies.
Maybe in this case something else might have changed assuming redirects are done correctly. How long it has been the redirects are functional. Also reason for redirect as there might be other things involved
Thanks for taking the time out to respond, what you say makes sense. We've started to see traffic turn a little albeit very slowly. Hopefully we are on the road to recovery – we would certainly go with your approach in future though about splitting into sections.
It'll take time around 2 months. The drop is just for short time, And the good URL structure will help you in long run.
Thanks for taking the time out to respond and for the timelines along what we should expect.
My Company has Extremely Similar WebSItes Should they be 301 Redirection or PBN? Private Blog Network
Best Practice of 301 Redirects
301 Redirect Is the Solution
How Cost did You Buy an Aged Domain for 301 Redirection?
Can Domain 1st Still Pass Pagerank to Domain 3rd if It had ever gotten Redirected to Domain 2nd for Over a Year?
Solution for TLD Domain Migration Down 40% Traffic, Is It Risk if It is Moved Back?