We check a new post after launch and it looks good... so we just assume the site's working perfectly. But is that a careless assumption?
It has been said that assumption is the mother of all f$%!-ups. It can be dangerous to assume that all is well, just because the page comes up in a browser. In fact, unsatisfactory performance in search engine rankings, traffic analytics and responses/conversions can often be traced back to a site that's broken.
Over the past sixteen years as a web developer and webmaster I've seen a lot of things that have broken websites. Here are a few of the classics from the hall of shame...
Sometimes the URL in a link was poorly formed in the first place and never tested. You of course always test every link when a page or post goes live, right?
An internal page or post may have been renamed or deleted, leaving internal broken links. Websites you linked to may have removed content, completed a redesign with a new platform, or disappeared entirely, resulting in broken external links. Websites can chew up a lot of hours in maintenance, and it's easy to think of these tasks as a necessary evil of providing content. That's when mistakes happen.
Not only do broken links make for a poor user experience; it's one of the factors Google's algorithm considers when ranking websites. If you're not checking your website regularly for broken links, there's no time like the present to adopt the practice.
Online tools like Broken Link Checker make it easy, and free. On the Mac, I like to use the Integrity app. For PC's there's Xenu's Link Sleuth. There's also a WordPress plugin I like that monitors your site's blog for broken links continuously: Broken Link Checker, with a list of any broken links detected in the Dashboard.
On CMS websites like WordPress and Joomla! installations, I've come across entire menus that suddenly broke. Yikes! Not one link or button worked. The web developer gets the blame for these, of course, but the cause has usually been traced to the renaming of categories, thus far, where the permalink includes the category alias. Unfortunately, the clients never considered the impact coming up with "better" names for things would have on the internal links, so there have been weeks following the updates where sites were completely unusable. A sharp drop in search engine rankings, or an email to the owner that their site was majorly screwed, were the first indicator that anything was wrong. In some cases, only one menu link will go bad, usually the result of altering the alias for a page or post, or removing content.
If you're a tinkerer, you should be clicking through your menus routinely. Note that many websites use JavaScript menus, and a few still have Flash menus, in which case broken menus will not show up in a broken link check.
One of the common causes for missing images is doing a bit of housecleaning in the WordPress or Joomla! media folder, and weeding out one too many files. Oops! This should be a very easy fix. If you right-click on the broken image icon in the web page, and check the info for the URL, you should get the filename and FTP directory it disappeared from. Your last backup should have the missing file. You do make regular backups of your root directory and take MySQL dumps, or click the backup button in the control panel your host provides, right? We like Backup Buddy in WordPress to automatically make backups of the database and files about once a week.
Here's one most people don't think of. Many hosting companies haven't switched to solid state (SSD) drives yet. That means hard drive platters spin nearly constantly and it's only a matter of time before the one your files are on fails. In an ideal world, a RAID mirror backup drive is immediately swapped in within a microsecond. Many hosting companies don't use redundant systems and instead have their hard drives backup during a slow period at night or, as we have discovered through hard experience, they may actually back up only weekly.
I recommend backing up on the server, but also having full recent backups on your own computer at all times. Eleven years ago we worked with a local hosting company that assured me that not being able to backup the MySQL databases with phpMyAdmin was not a problem. They had full backups on two servers and would be able to restore the sites within fifteen minutes if there was ever a problem. That worked just fine right until they went bankrupt, the servers were seized, and both the site files and backups for eight clients vanished. I lost tens of thousands on that episode, making it up to the clients, and still blush when I think of it.
Hard drive backups are not infallible and sometimes the most recent backup is actually corrupt. In severe data loss cases, your site won't load and all visitors will see is an error message. More frequently, you'll only discover missing images and other media days or weeks later. Some plugins may not work. Text may be garbled with weird symbols. Random weird stuff. In other cases, the company notices there are issues with the replacement drive and goes with an earlier backup. This can be serious if the content is time sensitive. For example, we built and maintained an online grocery store site that updated almost every day with new pricing and specials. To revert to a backup of eight days earlier caused some very costly issues in their ecommerce store, before we detected the problem and restored from our own backup, and we've learned to choose hosting companies more carefully for time sensitive applications like that.
Three times over the past decade another gremlin has wiped out database-driven CMS sites: timeouts during edit uploads. If you're making live edits to your WordPress, Joomla!, Drupal or other CMS site, a slow connection or server timeout during high traffic can not only cause the edit to fail, it can damage the MySQL database file. A few characters that don't transfer, are altered, or go missing is all it takes. With a corrupt database, nothing loads. Always test your edits in a browser, after refreshing your content, before walking away from the computer. Never assume that it looks great and that the changes took place without any glitches.
When WordPress or Joomla! asks you to make sure your site is fully backed up before you update the core software or extensions, please do as they recommend.
We've had updates fail completely, taking the site down within seconds. In other cases, only part of the site was damaged during a version update and we didn't know about it for days or weeks. Plugins and other extensions that once played nice together may have serious conflicts after an update.
Adding or removing an extension (plugins, modules and components, depending on the CMS) can cause problems. Do a quick check of the site any time you make a change. A few weeks ago, I discovered to my horror that the comments on my site had vanished... all of them. I began to backtrack in my mind and the only change I could think of was the installation of CommentLuv Premium. On a hunch, I deactivated it and instantly my comments returned. Very disappointed over loosing the benefits offered by the plugin, I tried reactivating it in WordPress. This time my comments were visible again. Just one of those funky things that happens sometimes when extending the functionality of a CMS website. Unfortunately, I went over a week without comments enabled because I failed to check everything immediately after making a change.
The 'white screen of death' is a common malady in WordPress. After updating plugins, suddenly all that shows in the browser is the infamous arctic whiteout scene. Failure to check your site after updating plugins could leave your visitors without a visible website for days or even weeks.
And then there's caching. Caching plugins can dramatically improve the page load speeds of a website, but they can cause the owner a lot of grief as well. If you use a caching solution that requires manual clearing of the cache, the changes you make may not be visible to your visitors. Most of these solutions won't show the cached version to the Administrator, so you'll be blissfully unaware that your revisions are not included in the cached version your visitors see..
Server timeouts or long download times for your visitors and search engine robots can dramatically increase your bounce rate, cause visitors to conclude your site's down (perhaps permanently) and affect rankings. If your website is temporarily unavailable, generating a 404 Page Not Found error when Google bots come by to index your site, this can severely hurt your search engine rankings.
The lowest priced hosting plan can be the worst deal out there. When we take on a new client, we frequently inherit a new host. Consequently we've dealt with more than 80 hosting companies over the years. The host we're putting most clients with these days is Liquid Web. I love "set-it-and-forget-it" hosting; the kind you never think of except when the monthly or annual hosting fee appears on your credit card statement.
It's very common for websites to slow down over a few months or years. More plugins are added to offer more features. Other software is uninstalled, but that leaves residual files. The Media Library keeps growing as images and video are added, but many are no longer even in use on the site. The database is clogged with references to software that no longer exists.
Slow page loads can seriously impact the user experience and search engine rankings. Periodically, it's a very good idea to evaluate your site's speed and clean up the extensions, files and database.
Have you taken a walk through your website lately, stopping on every page and post, to see what it looks like on a vertically held mobile phone? Both your users and Google rate your site on its mobile user experience. If it's not great, your site could be losing a lot of traffic, and may need some tweaks, or a switch to Google AMP for your mobile users.
Always, have a recent backup of your server's root directory files and database dump stored on your own hard drive or Dropbox! Never assume your site if still working perfectly. Check it for a few minutes almost every day, even if you haven't made any updates for a while. You may want to run a link check right now...
Have you ever had that sickening feeling where you asked yourself, "Oh my God, I wonder how long it's been like that?" Are there any tools you like to use for checking the integrity of your site or for automating backups? I welcome your comments and questions below.
Feeling a bit overwhelmed over your website, SEO, content marketing and social media? I'd be happy to answer any questions in the comments below, or feel free to schedule up a call.
Cole Wiebe helps brands and professionals grow their influence and value online; so they can “out content”™ their competition. Cole is a content strategist, content writer, conversion copywriter and online marketing coach. He is also a UX (user experience) web designer and developer of over 20 years.
So I’m not alone… Thanks for posting this.
A customer of mine told me that all of the pages and galleries in my site had been down for months. All I checked was the blog. Silly rabbit!
Hi Robert,
Thanks for the comment. I think it's happened to the best of us.
- Cole
Hi Cole,
Yes, I know the sickening feeling well lol 😉 On at least a few occasions. Check your links. Click through your site. Tweet old posts. Test, tweak, observe.
Actively retweeting or editing old posts can alert you to changes/broken links/broken images. Engaging your readers, fellow tweeters and other social media buddies also helps a ton.
If you have true friends, not just followers, a few will alert you to any broken aspects of your website. Such is the benefit of engaging frequently.
As for broken sites my blog has suffered the white screen of death on a few occasions after activating a plug in. The first time I had no idea what happened. The next few times it was usually a bad plug in. At any rate I have learned to get super familiar with my hostmonster tech reps when things go to blogging custard.
Thanks for the insightful share Cole!
Ryan
Hi Ryan,
Thanks for dropping by and taking the time to comment. "White screen of death"... I like that. I haven't been tweeting old posts, but that is an excellent idea for killing two birds with one stone: finding something to tweet about, while checking the dark corners of the site for problems.
- Cole
Hi Cole,
Well said on this mate. Now you really got me thinking. I was about to install broken link checker and saw you post on this.
Honestly, I can't stress enough the importance of making sure no 'bad links' on your blog. Good hate it and it is a no brainer to fix them.
Thanks for sharing this and appreciate it!
Hi Cole
I use the WordPress Backup To Dropbox plugin for backing my site up daily but it wouldn't hurt to take a backup to my hard drive as well.
I don't but I could regret that at a later date.
I was tinkering with some code yesterday and I caused my whole site to go down for a few minutes.
My heart was in my mouth but I managed to delete the offending code by ftp, get rid of the cached files and things were back to normal again! I must stop tinkering...
I regularly use the broken link checker online and that has been a great help to find any issues.
I hadn't thought of menus and I'd better go and check mine are all still working!
Hi Tim,
Sounds like suffer from the same affliction, the need to constantly tweak and tinker. I can't tell you how many times I've made a tweak, or installed a new plugin, added a snippet of code to the template and ended up breaking something on a WordPress site. I have to keep reminding myself that the site works just fine and provides a decent user experience. I've removed some of the resource hog plugins so it loads reasonably quickly again. Users aren't going to notice or care if I do anything else. I must take a full backup and then leave it the hell alone, and focus on content and interacting with the people that comment.
And then I see something cool on another site...
- Cole
Hey Cole,
Knock on wood but I've yet to have any mishaps with my blog where anything has been horribly broken.
I do backup my blog everyday manually, I don't trust any plugins anymore because too many have failed me. I backup to my hard drive, a program I use that backs up my hard drive and then my external hard drive. Yep I keep them in three separate places, better safe then sorry right.
I check for broken links regularly and I always test my posts. I never change my links after the fact so that's never been an issue for me. I have accidentally deleted images before but that was a long time ago before I actually had a clue what I was doing. Silly me.
Man, with everything you've mentioned here it's kind of scary all the different things that can go wrong. So much to think about right! Yikes!
Back up, back up, back up. It's always better to be safe than sorry.
~Adrienne
Hi Adrienne,
With a good backup, nothing's really all that scary, but it takes a trip 'round the site to check for anything that might have run amok. For a web developer, doing an FTP manual download of all of the root directory files by FTP is simple enough, and then we just need to go online to the hosting control panel, open phpMyAdmin and do a database dump. If trouble strikes, we just reverse the procedure... copy the files back to the server by FTP and import the former database version in phpMyAdmin. Fortunately, there's a much easier way if you're on WordPress. 🙂
With a WordPress site, we just install Updraft Plus - Backup/Restore. We purchase two in-app add-ons for Updraft Plus: "Backup more files including WordPress Core" and "Dropbox Folder". Once configured, Updraft Plus backs up the root files and database every night to Dropbox. If anything bad happens, Updraft Plus has the ability to completely restore to a previous backup. I have done a test restore, and it was perfect. I sleep a lot better knowing my WordPress client sites are being backed up every night like clockwork.
- Cole
Hi Cole,
We all know what they say about 'assuming' LOL! We really can't assume all is well all the time. It's important to stay aware and take precautions to prevent down time and broken sites.
It's amazing what you can find out if you simply go to your site as a visitor would and start looking around. I almost always find something to fix when I do that. But it helps to use good tools, like 'Broken Link Checker' to make sure things don't go awry.
One thing I'm pretty good about is backing things up. I've lost data in the past and it can be a sickening feeling. I use the WordPress database backup which backs up all my files, databases, etc and it does it automatically.
Liz
Hi Liz,
Thanks for stopping by and taking the time to comment. I can certainly relate to finding things to fix when I take a tour of my site. It's not always items that are broken, but content delivery that could have been handled so much better. For instance, I recently have seen two items on blog posts I love: "The takeaway" summary of the post and "Back to you" call to action. Also, placing the post photo to the right has been proven to encourage more reads. So, do I now go through all of my posts and update them? Tweaking and optimizing can be an endless process and taking a tour can produce and extensive list.
- Cole
Cole- When I give my site over to be handled by a web developer, I expect the site not to have broken links, work in all browser. If the category names don't work the developer should notify the client and make suggestions. I was dropping in ranking and found out that two companies, one in China and the other in Malaysia had copied my site verbatim. I rewrote all the text and I and starting to move up again.
The site is Copyscape.com which will check to see if anyone is copying your site.
Hi Arleen,
I agree that when a site is completed by a professional web developer, everything should work perfectly. In the instance I related, the categories were set up and all of the URLs worked on launch day. The client then went in and broke all of the internal links by renaming all of the categories. For example, http://www.realestatesite.com/listings/condos.html would have become http://www.realestatesite.com/properties/condos.html. Unfortunately, the menu item still linked to http://www.realestatesite.com/listings/condos.html. There's not much a developer can do to protect a site from destructive edits on the client side, and unfortunately we never suspected she would do that, so the problem was undetected for a while.
We've encountered the other situation you mention several times, where someone knocks off the content on another site. Copyscape is definitely invaluable in tracking sites like that down. It's unfortunate that the usual solution is revising the content on this end. The offending sites are usually foreign and feel they are out of reach for legal channels to be cost effective.
- Cole
Hi, Cole,
Broken links are really very painful as Google penalize websites which have broken links in their codes. Broken Links are also problematic for the user as they don't navigate through the site properly.
I used you check my all all links on regular basis and take all the back-ups. This is useful for me. Broken link checker can also help us a lot.
Thank you for sharing wonderful post. 🙂
Hi Kumar,
Thanks for stopping by and taking the time to comment. I'm glad the post was helpful and has inspired you to perhaps check for broken links more often.
Look forward to chatting with you again in comments,
Cole
Hi Cole
I love this...
"It has been said that assumption is the mother of all #$%!-ups."
So true.
It's so cool that there are people like you to help the rest of us who aren't as technical to know how to manage our blogs.
I do use WP Backup, and I also use Broken Link Checker.
Both have been very useful.
🙂
Hi Dana,
Glad you liked the quote. It comes from the movie "Under Siege 2: Dark Territory".
WordPress is certainly the best platform for bloggers that don't come from a developer background. Just like Apple... there's an app (plugin) for that.
- Cole
Hi Cole,
Great article and a real eye opener. As for having broken link issues...can't say that I've really had that happen on this site.
Although I did have one on another site. I was promoting an affiliate product and I had no clue that the affiliate owner had changed the link. I was promoting the wrong affiliate link for several months before I checked it out myself.
As for backing up my site.. I do daily backups with Backup Buddy. Seems to be working fine for me. I love it because it automates the process so I don't have to worry about doing it manually.
I just hope that if I ever do have to back up my site those backups work. Maybe I shouldn't rely 100% on a plugin. Thanks for sharing, have a great day.
Hi Susan,
I'm with you. I don't trust a plugin exclusively for my backups. The WordPress plugin handles day to day backups, to Dropbox, but once a week I like to download the site's root directory by FTP and take a MySQL database dump with phpMyAdmin, which I save to my hard drive. The hard drive is backed up to another one every night. I've seen too many things over the years, so I take backing up to the paranoid level :-).
- Cole
Hi Cole,
I'm glad I finally made it to this post that you had linked in your comment on my blog.
Great stuff, and it makes me think that I may have quite a few broken links I'm not aware of.
I try to check my links before I click on the publish button, but I do not doing it 100% of the time. Let's say a good 85% 🙂
What you're saying about categories is down right scary. One of the reasons I've never fixed my messed up categories yet, is the fear of what I don't know. This would be one of them.
When I go on my Google Webmaster Tool, I often discover that Google found some broken links which I "fix" there, but I'm not really understanding what this all means. Maybe you can clarify this for me.
Great post, thanks 🙂
Hi Sylviane,
Glad you stopped by. I'm not sure how you would fix broken links in Google Webmaster Tools, unless you are then going back to your site and tracking them down. I like the Broken Link Checker plugin the best. I just scroll to the bottom of the WordPress Dashboard and it lists any page, post or comment link that's broken. I can click directly on the source link provided to fix each item. It makes very short work of the process.
- Cole
Hey Cole,
So true my friend. So many things can and do go wrong. Seems you have been on the wrong end of it all. We do have to be careful not to go too cheap and also to check things and back them up. I am getting there, but not perfect wither. But having a good host (I use Bluehost and all good so far) seems to be super important.
I do have a little paranoia like Adrienne with my personal photos etc, with multiple copies and not keeping them at home when I am away. Too many bad things happen easily
cheers
ashley
Cole,
I am pretty good about backing up my website and checking it quite often for broken links. I keep a pretty close eye on everything, this is not to say I don't have issues but I am trying to avoid those catastrophes where I loose my entire blog.
I know that changing my categories is not a wise thing to do as it will cause broken links but I picked my categories back when I did not know what I was doing. Now I would like to go and fix my categories. Is there any way of doing that without messing everything up?
Very informative post.
Dee Ann Rice
Hi Dee Ann,
The site I mentioned in the post was a Joomla! site. The categories became part of the URL. ie: Changing the category "townhomes" to "townhouses" would change the actual page URL http://www.realestatesite.com/townhomes/123-any-street.html to http://www.realestatesite.com/townhouses/123-any-street.html. The menus did not automatically update to reflect those URL changes, which made many of the pages inaccessible due to the category change.
I have changed the categories in WordPress many times with no issues at all, so that's not something that you need to be concerned with architecturally, but you may want to do a link check if you ever make category changes, just in case you have some manually created links that were affected.
- Cole
I have faced these things in my blogs so now I take care of all these things like broken links and images, it effects our blog's ranking.
Hello Cole
Nice post. I think you have written very extensively about how anyone's website can be broken and how to overcome that. I will surely use these techniques on my blogs and see what happens.
Hi Cole,
Thanks for this here nice article and We all know what they say about ‘assuming’ LOL! We really can’t assume all is well all the time. It’s important to stay aware and take precautions to prevent down time and broken sites.