Imagine that there's been a tumultuous cut over from one website to another that's left you with over 6,000+ broken links ruining your search engine foo. As a sysadmin you can't fix the missing content but you can bring some peace to those 404s.
You programatically converted a report of those 6,000+ URLs into an Apache redirect file, reloaded Apache tested a couple of redirects manually and they were fine but you're not going to manually test those 6,000+ URLs are you? (the answer's "no", in case you're unsure).
I wrote broken_links.sh to address testing workload (available from my git repo or via github). When run, broken_links.sh asks for where the file of raw URLs is and where you would like any links that are still broken saved to.
$ cat /tmp/URLs.txt https://mcwhirter.com.au/ http://www.debian.org/ http://mcwhirter.com.au/broken.html
When you run broken_links.sh, your output will be similar to the below:
$ ./broken_links.sh Enter the path and file name that contains the broken URLs: /tmp/URLs.txt Enter the full or relative path & file name for the list of still broken URLs: /tmp/broken.txt Now checking: https://mcwhirter.com.au/ Now checking: http://www.debian.org/ Now checking: http://mcwhirter.com.au.au/broken.html Total URLs in the file: 3 Total URLs still broken: 1 /tmp/broken.txt
The output to the requested broken.txt looks like this:
Happy hunting still broken URLs :-)